------------------------------------------------------------------------------
  py2dx: Python to OpenDX Interface
------------------------------------------------------------------------------

   Version 2.0
   August 7, 2003

   Original
      Py-OpenDX 1.11
      Randall H. Hopper

   Modifications
      Kent Eschenberg (eschenbe@psc.edu) who is responsible for all errors
      Pittsburgh Supercomputing Center (http://www.psc.edu/~eschenbe)

------------------------
  Contents
------------------------

   Introduction
   System Requirements
   Installation
      Using Root Privledges
      Without Root Privledges
   DXLink Routines
   Usage
      DXL Interface
      Server Interface
      Starting DX
   Callbacks
   Usage of DXLInput
   Testing


------------------------------------
  Introduction
------------------------------------

OpenDX is a scientific visualization package that can be run interactively
using its own user interface or from another application using the DXLink
library of C routines. See http://www.opendx.org for more.

When run from the command line, DX consists of 3 processes:

   - the main script named dx
   - the user interface (UI) process named dxui
   - the executive process named dxexec

The DXLink routines can be used to communicate with dxui or dxexec. The Python
interface, py2dx, is basically a wrapper around the DXLink routines. A Python
program can start DX; load a network or define one on the fly; set and change
variables; and execute the network. The window in which DX displays its
graphics can be one created and managed by Python.

The interface has only been tested with Python programs that communicate with
dxexec while dxui is turned off. Python programs that communictate with dxui
may work fine.

---------------------------------------------------------
  System Requirements
---------------------------------------------------------

Testing of py2dx has been conducted with the following components:

   OpenDX 1.3 (http://www.opendx.org)
   SWIG 1.3.19 (http://www.swig.org)
   gcc 3.2.2
   Python 2.2.2 (http://www.python.org)
   Linux 9.0 (http://www.redhat.com)

It is strongly recommended that you NOT attempt to use py2dx with versions of
OpenDX earlier than 1.3.

You may not need to use SWIG at all. Please see the installation notes below.

Time and computer resources do not allow extensive testing with other
components. However, I will be happy to help get py2dx working on other systems
and to report the results and lessons learned in these notes.


------------------------------------
  Installation
------------------------------------

---
  Using Root Privledges
---

(1) Go to the Python extensions directory on your system. On Linux 9.0, for
    example, it is /usr/lib/python2.2/site-packages.

(2) If there is already a directory named DX, rename it (or delete it if you
    are brave).

(3) Create a new directory named DX.

(4) Download the py2dx package and store it in the DX directory.

(5) Uncompress and untar the downloaded file. This should create a subdirectory
    named Install that contains a number of files.

(6) Go to the subdirectory named Install.

(7) Edit Makefile if needed to set the correct locations for OpenDX and Python.

(8) Run make.

(9) Make creates many files. The following are the key ones that are read by
    Python to talk to OpenDX:

   In site-packages:
      DX.pth
   In DX:
      __init__.py
      __init__.pyc
      DXL.py
      DXL.pyc
      DXServer.py
      DXServer.pyc
      _DXL.so

(10) Regeneration of Some Interface Files

    It is recommended that you first try py2dx by NOT running SWIG. SWIG
    generates the files DX_wrap.c and DX.py from the OpenDX file dxl.h. The
    generated files are distributed with py2dx and may work just fine. Make will
    not think it needs to run SWIG as distributed.

    Note: Makefile renames DX.py to __init__.py.

    If your Python application seems to have trouble running OpenDX, check
    whether you are running versions of OpenDX or Linux that are newer than
    the versions listed above under "System Requirements". If your versions
    are newer, running SWIG to generate new versions of DX_wrap.c and DX.py
    may help.

    To run SWIG, first make sure that the version installed on your system
    is equal to or greater than the version listed above under "System
    Requirements". Then, move the old versions of DX_wrap.c and __init__.py
    to another location and run make.

---
  Without Root Privledges
---

If you do not have root privledges, alter the installation procees as follows:

   (A) Create a directory in your area named site-packages and, within it,
       another directory named DX.

   (B) Start at step (4) using your DX directory.

   (C) When done, ask your system administrator to execute steps (1) through (3)
       and then move all of the files in your site-packages and DX directories
       to the system version.


------------------------
  DXLink Routines
------------------------

The following routines are described in the DX documentation and are available
to Python.

   DXLStartDX                       DXLLoadVisualProgram     
   DXLCloseConnection               DXLLoadMacroFile         
   DXLSetSynchronization            exDXLLoadScript          
   DXLConnectToRunningServer        exDXLBeginMacroDefinition
   DXLSetMessageDebugging           exDXLEndMacroDefinition  
   DXLGetSocket                     DXLQuery                 
   DXLIsMessagePending              DXLSync                  
   DXLHandlePendingMessages         DXLGetExecutionStatus    
   DXLSetBrokenConnectionCallback   DXLExecuteOnce           
   DXLSetErrorHandler               DXLExecuteOnChange       
   DXLSetMessageHandler             exDXLExecuteOnceNamed    
   DXLRemoveMessageHandler          exDXLExecuteOnChangeNamed
   DXLSetValueHandler               DXLEndExecution          
   DXLRemoveValueHandler            DXLEndExecuteOnChange    
   DXLSetValue                      DXLSequencerCtl          
   DXLSetInteger                    DXLExitDX                
   DXLSetScalar                     DXLSend                  
   DXLSetString

In addition, a new routine, WaitForDXIdle, has been created to provide a way to
wait for DX to complete a task while also displaying errors and warnings which
may be created while performing a task.

By default, DXL errors are turned into Python exceptions. The exception thrown
is DXError with data being the error message. The Python try command can be
used to catch these errors if desired.


---------------
  Usage
---------------

Two schemes are currently provided for interacting with DXLink:

   - call the DXL interface directly
   - use the DXServer class (which calls the DXL interface)

---
  DXL Interface
---

This looks basically like C except that

   - DXL callbacks are written in Python instead of C

   - You don't need to check return values for errors 
       (errors are turned into Python "DXError" exceptions)

   - a few function signatures have changed

With DXL, you must keep track of the connection handle (from DXLStartDX)
and all the state data you associate with it so you can call the API. The
connection handle is merely an integer.

Example:

   import DX
   conn = DX.DXLStartDX( DX.DXEXEC + " -execonly -hwrender opengl ...", None )
   DX.DXLSetMessageDebugging( conn, 1 )
   DX.exDXLLoadScript( conn, "/my/path/DisplayTest.net" )
   DX.DXLExecuteOnce ( conn )
      ...

---
  Server Interface
---

This approach works a little nicer with Python. You create an instance of the
DXServer class and then call its member functions.The class has all the
features of the DXL routines plus:

   - Default error and broken connection behavior prevent application
       hangs when a DX error occurs

   - A slightly simpler interface; keeps track of DXL state data

   - Can create new objects that inherit the server

Example:

   import DXServer
   dx = DXServer.DXServer()
   dx.SetDXCommand( DX.DXEXEC )
   dx.Open()
   dx.LoadScript( "/my/path/DisplayTest.net" )
   dx.ExecuteOnce()
      ...

---
  Starting DX
---


If you want special options passed to DX or if you called your main dx script
something other than "dx", you should set the DXEXEC environment variable.
For example:

   setenv DXEXEC "dx"  (same as the default)
   setenv DXEXEC "dx -memory 64"
   setenv DXEXEC "opendx"


---------------------------
  Callbacks
---------------------------

This section can be skipped by the majority of users.

A "callback" is a Python routine that is called in response to a DX event. In
general, you will not know precisely when the event will occur (remember, there
are 3 processes involved). Callbacks may deliver errors, warnings, or values
that you requested.

By default, a couple of callbacks are established for important events such as
errors. Additional callbacks are created when you turn on certain features such
as DX debug messages. You may also establish your own callback for any event.

The py2dx interface includes some new C routines just to handle callbacks. These
are not merely wrappers around routines from the DXLink library. When py2dx
creates a callback, it creates a data structure that includes information on
the Python callback. It passes a pointer to this data structure to DXLink and
DX when the callback is registered.

A callback first shows up when the DXLink library (in C) calls a special routine
(in C) in the py2dx interface. This special routine gets back the pointer to
the py2dx callback data structure and uses it to call the desired Python
routine.

The Python programmer can blissfully proceed to register Python routines,
class member functions, and even lambda expressions as callbacks.


---------------------------------------------------
  Usage of DXLInput
---------------------------------------------------

Summary: it is usually better to set a variable with a command like

   DXLSend( "myvariable = 7;" )

than by using one of the DXLSet* commands.

---

This rest of this section can be skipped by the majority of users. These notes
apply to earlier versions of DX such as 4.1.3 but may not be accurate for later
versions.

---

DXLInputNamed and DXLInput are NOT functionally equivalent in all circumstances
in a DX network.

A difference comes up when you're in ExecuteOnChange mode.  If you
DXLSetInteger a DXLInput symbol for a .net-created DXLInput module, the
network does NOT automatically re-execute in response to the change.  If
you only use DXLSet*, you must force an execution in order for this
change to propogate (DXLExecute*Once), or simply wait for the next
execution to propagate it for it to take effect.  Not good.

However, if you use a DXLInputNamed module instead, setting its symbol
to a value via DXLSetInteger 'will' automatically re-execute the network
like it's supposed to.

THE HACK-AROUND: If you use DXLSend to set a DXLInput to a value, then
it will actually work.  So for every DXLSet* call, we both call the
appropriate DXLSet* call as well as invoke DXLSend to set the value of
the script variable with the same name.  This covers both cases
(i.e. where the module is a DXLInput, and where it's a DXLInputNamed).

As to the robustness of this hack: Note that every DXLInput actually
materializes as a script variable of the same name in the DX network
introduced by both DXLSending and DXLSeting the value.  In fact, DX is
going to set this variable anyway so you might as well.  However, if the
module is a 'DXLInputNamed', it doesn't materialize as a script
variable.  So concievably there might be a conflict by setting a script
variable of the same name using DXLSend.  The question is whether DX
allows both a DXLInput and a DXLInputNamed which both answer to the same
name.  Probably not since DXLSet can set both; either way, it's perverse
to do this.  So by default our DXLSet* wrapper calls do both DXLSet and
DXLSend for Set* wrapper calls.

If you directly set the output script variable for the 'virtual' DXLInput
module (e.g. main_DXLInput_57_out_1 below) via DXLSetInteger, it still won't
execute.  However, different than before, it 'won't' get picked up when you
execute the net later either.

DXLSetValue doesn't help you any over DXLSetInteger

Interesting related note: When you do a DXLSet*, the DX message you get
back across the wire explicitly says it's a SetDXLInputNamed event:

    <DX Msg> INTERRUPT :  begin /SetDXLInputNamed:0
    <DX Msg> INTERRUPT :  end   /SetDXLInputNamed:0

Another interesting note (possibly related): In a .net file, notice that
a DXLInputNamed module gets an explicit module definition:

       // 
       // node DXLInputNamed[3]: x = 4299, y = 424, inputs = 2, label = \
               DXLInputNamed
       // input[1]: defaulting = 0, visible = 1, type = 32, value = \
               "InteractorMode"
       // input[2]: defaulting = 0, visible = 1, type = 29, value = 0
       // page group: Main
       //
   main_DXLInputNamed_3_out_1 = 
       DXLInputNamed(
       main_DXLInputNamed_3_in_1,
       main_DXLInputNamed_3_in_2
       ) [instance: 3, cache: 1];
       // 
       // node Receiver[277]: x = 392, y = 98, inputs = 1, label = \
               PickIsActive
       // page group: IRR_MB
       //

whereas DXLInput does not.  It only gets a script variable placeholder to
transfer to the output name of a 'virtual' module:

       // 
       // node DXLInput[57]: x = 4301, y = 394, inputs = 1, label = \
               InteractorMode
       // input[1]: defaulting = 0, visible = 1, type = 29, value = 0
       // page group: Main
       //
       main_DXLInput_57_out_1 = InteractorMode;
       // 


---------------------
  Testing
---------------------

These programs were used to develop and test py2dx. They may not be the best
example of how to use DX from Python. Some have system-dependent features and
so may not work on all systems.

   testServer.py           - Test the DXServer class.

   00-basic.py             - Fire up dxexec and shut it down

   01-run_file_net.py      - A really simple display test.  Loads a .net file,
                             runs it once, and then quits.

   02-run_dynamic_net.py   - Similar to 01-run_file_net, but creates the
                             network on-the-fly via DXLSend, rather than
                             loading a canned network from a .net file.
                               
   03-tk_win.py            - Moving closer to doing something useful.  This
                             test program sets up DX to display images 
                             in a Python-created Tkinter window.
                             
   04-value_hdlr.py        - Very simple test of value handlers.  When a
                             DXLOutput value changes, DX calls value handlers.

   05-connect_hdlr.py      - Same thing, but with connection handlers.

   06-msg_hdlr.py          - Same thing, but with message handlers.

   07-hwrender.py          - PYTHON-DX COLOR CUBE BENCHMARK.
                             Displays a spinning cube using various rendering
                             mechanisms (opengl, gl, software).

   08-userctl.py           - Mouse control of a DX window.
                             (Hardware-window interaction only works with 
                             a patch to DX).

   09-userctl2.py          - Mouse control of a DX window.

   10-supervise_spin.py    - Test of SuperviseWindow/State modules for 
                             interaction

   11-supervise_net.py     - Similar, but with a network loaded from a .net
                             file rather than one that's dynamically created.

   12-userctl3.py          - Another user interaction demo in a Tk window.

   13-userctl_gui.py       - DX-in-a-window test.  User selection of
                             software/hardware rendering, interaction method
                             (both software and hardare), resizable GUI, etc.
                             Uses the DXServer interface.

   14-value_relay.py       - Simple test of chained network executions.

---------
  End
---------
