-*- outline -*-
*Experiments with ZUIs

**DRAWING

Piccolo does the drawing by letting each node draw into its local
coordinate system, and manipulating the (java.awt) graphics that it is
given.  The current context is available to the node when it is
rendered; it contains, among other things, the stacks of cameras,
transforms and clipping rects to that point in the scenegraph.

Jazz works much the same -- essentially, Piccolo just collapsed a lot
of the common functionality into the top-level node class.  Pad++ had
a few optimisations over these guys, but I think mainly because there
was no decent drawing library to fall back on.  In Jazz and Piccolo,
the rendering and everything else is based on the scene-graph; In
Pad++ (I believe) they were more orthogonal; e.g., it used a spatial
index for rendering, and a centralised model for damage reporting.

Without a transformable drawing context scale is tricky.  We can
either render to surfaces and interpolate, or we can let the node
itself decide how to render itself.  A mix might be best: have render
and render-to-local.  The default is to call render-to-local and
transform; however, subclasses can always override render and account
for the transform themselves (does alpha also work this way, or should
we always composite it?).  Luckily Cairo gives us exactly such a
context, and we can save and restore the state to support scenegraph
walking.

Squeak/Self?

**DAMAGE

Jazz and Piccolo work in more or less the same way w.r.t. damage:
nodes register damage with their parent giving a rect in local
co-ordinates, which is passed up the scenegraph until it reaches a
camera, which is repainted.  Each layer maintains a list of cameras
through which it is viewed so it can notify the camera.  In Pad++,
each camera maintains a list of other cameras visible through it.
Aside from a few tricks with blitting, rendering is (unsurprisingly)
similar to Jazz and Piccolo, with the difference that damage is
registered with repainters, which can coalesce areas.  Repainters also
have levels of refinement -- typically a repainter will paint, then
register the same area for further refinement.

Roughly damage operates through a pair of methods: [code
invalidate-rect] is called by a zorph when its model has changed or
some other condition means it should be redrawn; [code notify-damage]
is called to notify an observer that something has been invalidated
and an area needs to be redrawn.  [code invalidate-rect] is done in
local co-ords, so the rect must be transformed into observer
co-ordinates before passing on.  There is a variation on notify-damage
that deals with things being viewed (rather than contained) --
primarily this is so cameras can transform the damage rect using their
view transformation.

In a traits-y way, we can delegate both the transformation and the
list of things to notify to the node itself as prerequisites.  Layers,
for example, will want to notify both their parents and the cameras
that look at them.

Squeak/Self?

**COMPONENTS

Interactive components in Piccolo are conflated with drawing
primitives (at least, they inhabit the same inheritance hierarchy and
scene-graph).  I think it best to conceptually distinguish between
things that are just drawing primitives and interactive things; it's
handy to reuse the rendering protocol for both, so we can keep them in
the same scene-graph, it'll just be one built of components with
sub-trees of primitives 'hanging off' those.  Components are really
just views, and they will have all the MVC machinery.  Zorphs are also
a way of adding default functionality; in the same way, for example,
that morphs in Morphic have halos.

Piccolo adopts a similar strategy to Java Swing, in that it collapses
the view and controller together and occasionally has adapters.

Squeak/Self?

**INPUT EVENTS

Here is something that is fairly non-controversial: input events
should be given to whatever is the user's locus of attention.
Interpreting what that is may be harder, for a few reasons:

1. There is typically more than one input device.  The usual approach
is to make keyboard input modal; i.e., the locus is at the mouse
unless the user is editing some text.  This introduces the notions of
an 'active component', 'cursor', etc.  It's not unreasonable to expect
that a mouse event means the locus is at the mouse, but a keyboard
event is trickier.

2. Components can overlap and contain each other, requiring some kind
of delegation.  Different implementations go to different lengths to
constrain this: in Smalltalk, it is simply the default implementation.

3. There is inevitably some guessing as to the user's intent, which
result in heuristic semantics.  For example, most scroll bar
implementations keep the input event focus while the user is dragging
the slider, even if they stray outside the component bounds.  This
complicates the protocol again.

***Hierarchical delegation

In this model, components are in a tree representing spatial
containment.  With a hierarchy, the usual approach is to allow the
outer-, topmost container to handle things and pass unwanted events
on.  This requires a couple of refinements of the protocol: firstly,
to signal whether an event has been handled, and secondly, to decide
on which order siblings are given the event (spatially, this is
normally the z-order).

***Pattern-matching

Perhaps a more general model is to throw events onto a central
bulletin board and let components (or indeed, whatever) pick them out
by pattern-matching.  Note that pattern-matching includes spatial
constraints, so in principle hierarchical containment can be encoded;
however, because this is inherently a free for all (with no
communication between components) delegation isn't so simple.

***Keybindings

For keyboard events, we really need a stack that keeps a track of
which modifiers are down and in what order the keys were pressed;
keybindings really crys out for a DSL.

I like the Emacs model of /everything/ being a keybinding -- perhaps
with a 'self-insert' default -- but it needs to be extended to account
for more Raskin-y key sequences, like <shift>/ a/ <shift>\ a\ (press
shift down, press 'a' down, release shift, release 'a')

I guess you could keep keybindings in a prefix tree and walk it to get
the command.  Don't know how defaults work in this case (perhaps can
tag for this case, and be able to extract the character).

I don't like the idea of modes in the keybindings, but it may be
inevitable -- for example, if one is editing text, necessarily keys
are interpreted differently (even if only slightly differently).
Perhaps the keybinding tree can be merged or shallowly cloned.  We can
intercept certain key presses before they get to keybindings.

The other interesting thing is how these interact with other gestures
-- modifier keys might be desired for mouse gestures, for example.


**BASE MODEL

Eventually the Zorphs and so on must be reflective and tinkerable.
There's only so much of this that tinyclos can do, and no rewiring
like Self.  Perhaps this will come cheaply when it is rewritten in a
prototypical, reflective language.  We may be able to produce a
browser using tinyclos's reflective capabilities with a suitable layer
of shielding over the top.

In the MVC/Morphic model, the database is the models referred to in
the scenegraph; the data is implicit in the environment.  Another
approach is to have the database explicit and refer to it.  The
scenegraph becomes a mechanism for the view, at a remove from the
model; adapters hook the immediate models of the views into parts of
the database.  There is a real separation of model, view and controller
here, and much less of a notion of encapsulated widgets.

**STYLING

As much as I despise the skinnability red-herring, I believe that
shiny styling can be a useful draw for users.  I have only vague ideas
for this at the minute.  I wonder if there is an abstraction for
styling that 1. doesn't upset the rendering protocol too much; and
2. admits several quite different approaches.

***Cascading stylesheets

In an interview [1], one of the progenitors of the Newton mentions
Bahaus having '.. nestable stylesheets for describing the appearance
of UI elements (sort of like CSS for UI ...)'.  That sounds like an
intriguing prospect.

***Pluggable look and feel

A look and feel a la Java Swing might be more high-level; it is
concerned with how specific widgets look, rather than how drawing
primitives look.  It could be extended to widget behaviour and layout:
for example, Apple menus vs Windows menus.  I think this kind of
styling typically uses a toolkit pattern (i.e., a singleton that
supplies specific implementations of an abstract class hierarchy).


--------------------------------------------------

[1] http://lispm.dyndns.org/lisp/mikel-evins-on-newton.text