Inductive Groups - ACM Digital Library
Inductive
Groups
Dan R. Olsen Jr. Xinyu
Deng
Computer Science Department Brigham Young University Provo, UT 84602 [email protected],edu
ABSTRACT The notion of inductive for
manipulating
behavior
groups is presented as a mechanism
sets of related
objects.
The interactive
for
discovering
general. An application
inductive
relationships
The work complements free form
INTRODUCTION The notion of a group
software,
inductive
form of such inductive
is found in Peridot’s handling
discovery
of lists [Myers92].
in
using these techniques is shown.
KEYWORDS: user interface interaction by demonstration.
of the group. Based on these relationships, behaviors which leverage across the entire group
can be defined. A limited
of such groups is discussed along with extensible
algorithms
definition interactive
groups,
has long been a part of drawing
graphics
the work on discovering [Moran95,
Shipman95].
structure in Their
work
attempts to discover groups and relationships among groups. In this work we take an already recognized or specified group and discover regularities in the group. Based on these regularities we provide more amplified interactions
with the group itself.
packages. The set of things that can be done with a group of objects and the ways in which those objects interrelate been quite operations scaling.
limited. allowed
Any
other
In most drawing
packages
on the entire group are translation operations
require
that the group
broken apart and the objects manipulated exception groups.
to this is Aldus Intellidraw’s
We are interested in more flexible because they across a whole
allow
independently.
Metamouse [Maulsby92] the interactive behavior actions. This produces
and be An
notion of distributed
representations
user manipulations
set of objects,
has
the only
for working on widgets
with groups by example stems by
example
pictures as input data [Kohlert95]. we only had learning finite set of relationships.
[Olsen95]
and
In both of these systems
mechanisms
which
could
handle
a
Having done that work we wanted
of groups
to be leveraged
or Eager [Cypher93] which watch of the user and infer repetitive which
our work
to extend those mechanisms to lists of arbitrary length. The inductive groups presented here are a first step towards that goal.
There are systems such as
procedures
Our motivation from
can generate or
BEHAVIOR
OF INDUCTIVE
GROUPS
Before discussing the implementation is useful to consider their behavior.
of inductive
groups it
Take for example
bar chart shown in Figure 1. There are five inductive in this chart. They are the bars themselves,
the
groups
each set of tick
modify groups of objects. Such techniques, however, do not preserve the discovered relationships between objects and
marks on the two axes, the numbers on the left side and the
must be redemonstrated
these groups carefully one can see constant properties that all members of the group share, inductive properties (where the property for the i-th object can be computed from the property of the (i- l)th object) and variable properties that have no relationship at all to the other objects.
with each new activity.
In this paper we discuss the notion an inductive group, relationships discovered
by
example
of inductive groups. In between objects are
and maintained
as part
of
the
Permission to make digital/hard copies of all or part of this material for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication and its date appear, and notice is given that copyright is by permission of the ACM, Inc. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires specific permission and/or fee. UIST ’96 Seattle Washington USA @ 1996 ACM 0-89791-798-7/96/11 ..$3.50
days of the week across the bottom.
Let us assume for our discussion
By considering
that the black
each of
bars are
rectangles which have as basic properties fill color, width, height, and lower left point. If we consider these bars as a group we see that fill color, width and lower Y coordinate are constant across all of the rectangles in the group. The X position of each bar can be determined inductively by adding a constant to the X position if the preceding bar. The
193
height
of each bar is completely
discoverable
relationship
variable.
There
is no
among the heights.
Changing a constant propetiy Suppose that the user changed the color of one of the bars in our example. The color is a constant property
The bars demonstrate An inductive all
object
computed properties properties
group can have constant properties
of the group,
inductive
properties
some function
for computing
Changing the color of one bar will change the color of all. If
shared by
the user was to grab the lower end point of one of the tick marks in Figure 2a and drag it downward, this changes the
which
a value from preceding
sequences can be characterized
value, induction function. The remaining
function
and
the
are
values.
by the initial
coefficients
of
that
Y position which is a constant property of this inductive group. The result is to lengthen all of the tick marks as shown in Figure 2b. Changing a constant property of any object changes that property for all objects in the inductive group. Note that the issue of which properties determined
groups in our bar chart further
of inductive
The
groups.
induction
because it recognizes incremented interesting.
by
illustrate
The horizontal
on the
string
row
values
the
days
of
the
themselves
follow
week
and are are more
constant or inductive an inductive
L II
al
of tick
sequence
through the names of the days of the week.
‘k‘
b
I
Figure 2: Changing constant properties
is different
that the strings are numeric
10. The
Not only is the geometry
but the strings
are constant is
at the time the group is created. This creation
process is discussed later in the paper.
marks have all of their properties constant except their X coordinates which are inductive by adding a constant. The numbers on the left side have a constant X coordinate and font, an inductive Y coordinate and an inductive string value.
of the group.
group.
by some inductive function and variable which are unique to each object. The inductive have their values defined by an initial value and
Such inductive
concept
what we mean by an inductive
Changing inductive properties If the user changes an object property induction,
which
is defined
there are two special cases to consider,
value and some induced
by
the base
value. Suppose that the left-most
tick mark in Figure 3a was moved to the right. This would change its X coordinate which is an inductive value. The left most tick mark, however, is the base or initial object for this inductive group. Moving the left-most tick mark would change the initial value of the induction. In the case of this
30
example,
20
right
right-most shown
10
coefficient
M3ncky Figure
T ues day 1:
Wednesduy
Bar Chart
Interactive behavior Having recognized and created inductive
groups, there are a
variety of interactive behaviors that can be defined on such groups. All of these behaviors are defined in terms of manipulating a single element of the group. manipulating a single element a user can leverage behavior across the entire group.
By that
When a user manipulates an object on the screen, the application is changing some of the model properties of that object. If one drags one of the bars, the X and Y of the lower left corner are changed. The impact of such changes on the entire group is different for constant, inductive variable properties in the group.
~nd
194
set of tick marks would
move to the
that the last or
tick mark in the group was moved to the right, as
in 3c. The X coordinate
induced right
the entire
as shown in 3b. Suppose, however,
value.
The
behavior
of the induction
of this tick then
is
to
mark change
is an the
so that the induced value of this
most tick mark matches the value that the user has
requested for this object. In this case the other tick marks would be repositioned to reflect the new coefficient as shown in Figure 3d. Note that changing any other of the tick marks would have a similar effect. The user would change the one that was most meaningful to the task at hand, with the induction function working out what needs to be done to accommodate
the user’s request.
Similarly in Figure 1, if the user changed Monday to Sunday, then the sequence would change to (Sunday, Monday, Tuesday) because Monday is the base value for this inductive group. If instead the user changed Tuesday to Wednesday, then the sequence would change to (Monday, Wednesday, Friday) because the induction coefficient would change to 2 days instead of one. If the user were to change the number 30 to 50 then the sequence of numbers would change to (10, 30, 50). The coefficient of 10 between each number is changed to 20 which is what is needed to
accommodate
the user’s request that the third item be 50.
This behavior changes
to
flexibility
of inductive individual
properties
objects
in the way
in response to user
provides
a whole
that users can manipulate
new
sets of
calculate
the inductive
inductive
functions
and coefficients
of the
properties.
Any properties which cannot as constants or by induction are retained
explained
be as
variable properties.
objects. Brief NIC introduction Our implementation of inductive groups was done in the Nucleus for Interactive Computing or NIC system [NIC]. There are a few aspects of the NIC architecture
which
are
important to understanding how inductive groups are implemented. The key issue here is the way in which NIC models interactive
data. Similar
techniques
can be used in
other systems.
Figure 3: Changing
Inductive Properties
Changing variable propeflies If the user was to change the height of the middle bar alone would
be changed.
of
bar, that
following
the deleted
object,
their position
are
already
properties properties
is added into the group,
defined
by
the
group
are
because
important
because their position
in the inductive
inductive
induction
sequence
DISCOVERY Having
which
can be inserted
how
the constant
properties
of
the group
and
characters
of an object are identified
indices
support
a variable
number
of
or index of a NIC object
value or another NIC object.
The most
aspect of this for our purposes is that it is possible
discovery
and object
manipulation
algorithms
from which to form the group is created from an ordered set of
In most cases the order is quite natural to the
and does not require
any special remembering
of
the order.
behave
interactively we can now address ourselves to the actual creation of inductive groups. This is performed by example. A user selects an ordered list of objects. The first object is considered to be the base or initial object on which the group is based. This example set of objects is used to determine
values
longs,
to be performed. We do not believe that this ordering problem is a usability issue once users understand that order
and
problem groups
floats,
objects. A user uses standard selection mechanisms such as “shift-click” for selecting multiple objects one at a time. The order in which the objects are selected is important because that forms the sequence across which induction is
GROUPS inductive
are primitive
Objects have a class, zero or more attributes and
is important.
OF INDUCTIVE
discussed
elements
there
values include
Sequence of objects An inductive group
object copies the variable properties of the object which precedes it in the group. The user is then free to change them as desired. This addition and deletion feature allows inductive groups to take on the behavior of semi-uniform into
data model
Primitive
which can apply to any object in the user interface.
has changed. The variable properties of the new object are undefined. As a convention in our implementation the new
lists of objects deleted.
data
at run-time to identify and retrieve all of the component values of an object. This makes it possible to write general
can be computed from the group. The inductive of all objects following the new object must be
recomputed
manipulating
model and an extensible set of induction functions. Thus the mechanisms are the same for lines, rectangles, days of the
objects that support
of the object the
and
in terms of a general
them. The value of any attribute
it is added after
and
discovering
in a wide variety
packages. For this reason all
discovering
are defined
can be a primitive
object. The constant properties
that we define for
by a symbolic name and indices by an index varying from zero to the number of indexed values in the object. Most
properties remain the same.
some existing
for
zero or more indices. Attributes
in the sequence has changed and their variable
When an object
groups
and NULL.
group only the
are recomputed
not just drawing
mechanisms
inductive
objects.
their constant properties
properties
architecture
groups which could be applied
In the NIC
it in the group are changed. For all objects
the same, their inductive
the
it was very important
a general
week or tasks in a Pert chart.
at the time the
Adding and deleting objects When an object is deleted from an inductive objects following
and
of situations,
The heights of the bars are
was discovered
this work
algorithms inductive
variable and thus unrelated to each other. The height of each bar functions independently. The fact that bar height is variable across this group group was created.
In doing
In our example the user might first draw the three black bars as in Figure 1. Having positioned the bars as shown in the figure, the user can then select them in left to right order. This makes the left-most bar the base for the inductive sequence. Having selected the objects, the user requests that
and to
195
an inductive
group
be formed
either
using the menu
or
accelerator keys, Discovery of constants and variables Once the user has presented an ordered list of objects to be formed into an inductive group, the first differentiate between the constant and variable
step is to parts of the
objects. Selection of our example black bars would produce the list of NIC objects shown in Figure 4.
[Width
20]
[Width
20]
[Width
c
[Height
501
[Height
20]
150]
[Height
1001
induction functions are dates, months, log scales and various string inductions such as (XO1, X02, X03) or (Va, Vb, Vc). The lnductFunct abstract class The induction discovery algorithm
must be generalized
broad class of induction
In order to accommodate
functions.
to a
this, we define the abstract class InductFunct. All of the inductive group algorithms are based on this abstract class.
[Color
(O O 0)1)
[Color
(O O 0)1)
[Color
using string representation of numbers, and induction on strings which are names of days of the week. Other possible
Our implementation
of inductive
subclasses of InductFunct. registry
(O O 0)1)
groups has quite a number
These classes are placed
where the general algorithms
makes the addition
in a
can find them. This
of new forms of induction
very easy.
Figure 4: Selected list of NIC objects The InductFunct abstract class has three methods that form the primary interface to the rest of the system.
The first step is to compare all of the objects to the base, or first, object. This comparison is recursive and will identify all of the components components pattern
which
are replaced
shown
portions ofthe
are different.
by variables.
in Figure
This produces
– This method
CalcCoeff
These different
and
argument
the
[X
5. Note that all of the constant
objects have remained,
~Width
!Vl]
[Y
201
between
10]
~Heiaht
!V21
[Color
Figure 5: Pattern generalized
1
(O O 0)1)
a value
implementation
from examples
the
fuzzy
form
corresponding
is different
from
the
base.
In
a variable
is
introduced
only
values are more than a little different.
that are close to each other are considered
if
variables
the and
This
{ {
50, 50,
Figure6:
150, 150,
250 100
Sequences
how
close
the
calculated
method
takes
an index
into
the
is the
properties
or initial
value so that
have the new value way
in
which
the
specified.
changes
to
are handled.
– This takes an element index, calculates the value of that element using the stored coefficients and
BuildElement
returns that element. This is the method for regenerating the sequence from the inductive function.
from
To better understand how these methods work, consider the We can first create a sequence of values for !Vl. DaysOfWeek
induction
function
object.
We
can
call
CalcCoeff on the DaysOfWeek object passing in the sequence { 50, 150, 250 }. The result will be 0.0 because DaysOfWeek expects a sequence of day names. We next try an Additive induction function object. Its CalcCoeff method
6. These values are
} }
This
value will
method
inductive
Inthepattern generated in Figure5 there are two variables, !Vl and !V2. Each of these variables ~e associated With the
: V2
–
the indexed
which are truly variable.
!Vl
1.0 for
This will change the coefficients
constants. Our next step is to determine which of the variables can be described by an induction function and
sequences of values shown in Figure drawn from the example objects.
and
sequence and a new value for the element at that index.
the base value is retained as a constant. This fuzzy comparison allows for the fact that users cannot draw their examples exactly. Discovery of induction variables The previous step has differentiated
For values that are close, it returns a result 0.0
ChangeCoe#
In
Values
equivalent
induction
coefficients are stored in the induction function object. This object can now be used in place of the original example list of values.
our
we used a fuzzy form of this comparison.
calculate
induction is to the example values. This method is the heart of the induction discovery. The calculated
This generalization from examples to a pattern with variables has two forms. The crisp form creates a variable whenever
to
coefficients which can generate that sequence. This method returns a 1.0 if it was exactly successful and 0.0 if it failed.
(:Rect
takes a sequence of values as an
attempts
returns 1.0 and stores the initial value 50 and coefficient 100 in the function object. The additive induction function
I
exactly explains the sequence of numbers. If the sequence for !V2 { 50, 150, 100 } were passed to an Additive induction function it would probably store 50 as its initial value and 25 as its coefficient but would return 0.2 or less
ofvalues
There are many possible induction functions that could be used, In our example we use an additive function, additive
196
as its result because the induction
generalize foreach
does not explain the value
very well.
(I*coe~
method actually
For example + init)
the Additive
calculates the induction induction
function
HF
= null
HC
=
is
where I is the index of the element. Given
return 50, 150 and 250 respectively.
The
method
properties.
most bar in Figure position
P
This reconstructs
supports
the
changing
of
?
Suppose that the user moves the right-
function
explained
by
HF
}
object that is
as the new value. The Additive function object would recalculate the coefficient to be 105 instead of 100. By
}
HC > threshold then V is an induction property else V is truly variable
the bar at 260 instead of 250. The system would on the Additive
F
if
1 to the right by 10 pixels. This would
invoke ChangeCoeff
sequence S for from the examples of induction function
create an induction function object NC = F. CalcCoeff (S) note that F now contains the ca lculated coef ficients if NC ~ IiC then { HF=F HC = NC
the
describing the X position of the bars. ChangeCoeff would receive 2 as the index of the element being changed and 260
V
L
sequence.
ChangeCoeff
inductive
pattern
0.0
compute value variable foreach class r
the induction function discovered above. Calling BuildElement on that object with indices of O, 1 and 2 will inductive
to form in P
{
The BuildElement function.
examples variable V
It is important
to note that although
the general matching
algorithm is recursive, the check for inductive is not. The cost of this check is
calling BuildElement we can reconstruct the new sequence as 50, 155 and 260. These reconstructed values can be used
NumberOfFttncs
relationships
* NumberOfVars
to redraw the bars in their new locations.
Representing
In the case of the DaysOfWeek
The information about constants, variables and inductive variables that is discovered by these algorithms is assembled
process induction
is
similar
to
that
class, CalcCoeff
subclass of InductFunct, described
for
the
the
Additive
will attempt to find a sequence of
strings that are ordered by days of the week and will store a start day and a days-to-increment as its coefficient. BuildElement will use a modulo function and a table lookup to return the day name for elements in the learned sequence. ChangeCoeff can be used to change the starting day or the increment to be used. These three methods provide for a wide variety of inductive
behaviors to be represented.
together in an InductiveGroup object supports interaction the interprets
that will
for each variable.
explain
the sequence of example
This discovery
possible to create several inductive
function
groups and then group
group.
SPACES
situations
the relationships
appear to be quite complex.
Consider
among
objects
for example
can
the tick
marks around the edge of a dial shown in Figure 7.
values
creating an object method using the
example values. The function which returns the highest confidence is retained and if that confidence is above the inductive
objects and
as changes to the inductive
process consists of taking
each registered subclass of InductFunct, of that class and trying the CalcCoeff
threshold
that interaction
them as a larger inductive PARAMETRIC
Induction discovery algorithm The induction discovery algorithm consists of discovering the variables and then trying to discover an induction
object. The InductiveGroup
with the constituent
description of the group. The inductive group that is assembled using these algorithms its itself a NIC object. It is
In many
function
the group
I
object is stored in place of
I
the sequence of example values. If more than one function is above threshold, then the function that returned the
Figure 7: A dial
highest vah,te is chosen. If there is more than one function with that same highest value, then the first one in the list is
The tick marks are evenly spaced around the edge of the dial, but in Cartesian coordinates their inductive relationship is sinusoidal, requiring several coefficients and thus 5 or
chosen. The implementor can order this list to select the most likely functions first. The full algorithm is as follows.
more examples to learn the inductive
function
correctly.
As in our work on widgets by example [Olsen95] we simplify the problem using parametric spaces. In our drawing tool, objects are drawn on top of other objects and use the coordinate systems defined by those objects. The
197
background
has a standard rectangular
The base of the example coordinate
system.
dial,
The tick
coordinate
being
marks,
system of the arc can be explained
system.
an arc, has a polar using
problem,
however,
is that after inserting
the first tick mark,
the result is as shown in Figure 9.
the coordinate
by the same inductive
functions used for the tick marks of the bar chart which are in rectangular coordinates. Our drawing relative
to
system defines parametric rectangles
(normal
Cartesian), parallelograms (polar) and splines (“curvy” offered
objects
will
lines
[Lin93] (angled
(oblique coordinates), arcs coordinates). The affordance
to the user by this model
‘circular’
coordinates
Cartesian),
function
is that “drawing
in a ‘circular’
way
on and
drawing on ‘curvy’ objects will function in a ‘curvy’ way”. We believe (without experimentation) that this is a naturally understood
mechanism
for
expressing
relationships.
\ Figure 9: After tick mark insertion
The
result is that the induction discovery process is greatly simplified and the number of examples required is reduced.
Inserting
the tick mark moved all of the others around. The
user can grab the stray tick mark and drag it back to the ending INTERACTION Having
WITH INDUCTIVE
defined
the
groups we would
algorithms
like to revisit
and show how the machinery take for example Figure 7.
GROUPS for
managing
their interactive
accomplishes
the creation
position
where it belongs.
Dragging
mark back will cause a ChangeCoeff inductive
recalculate
behavior
The user can therefore
the goal. Let us
of the dial
tick
marks
in
the coefficients
and reposition
the tick marks.
insert the desired
number
of tick
marks and then drag the last one to its desired location. The result is the evenly spaced set of tick marks in Figure 7. The example
of the dial tick marks is a form
Suppose that first our user drew three tick marks as shown
insertion
in Figure
extend indefinitely
8. Because they are drawn on an arc they are in
the stray tick
method to be called to
of bounded
where the user does not want the list of objects to but rather stay within
the bounds of the
polar coordinates (a,r) where a is the angle and r is the radial distance from the center of the arc. The user can then
arc. To accommodate this we have provided explicitly for bounded, as opposed to free, insertions. When a bounded
select these lines
insertion
in left
to right
order
and request
the
formation of an inductive group. The color and line width of the lines are the same and will become constant properties of the group.
is performed,
we first copy the values of the last
the object in the induction sequence. After performing insertion (or deletion), we use ChangeCoeff to put the values of the last object back to what they were before the insertion. This has the effect of locking first object coefficients problem
both the last and the
in place and then adjusting appropriately. Using bounded
shown
in Figure
9 would
Instead after the first insertion, as in Figure 10.
not
the induction insertion, the have
the result would
occurred. have been
Figure 8: Example dial tick marks
Looking carefully at these tick marks we see that their end points all share the same r coordinate. These too are constant properties. The a coordinates of the end points, however, are all different and will be generalized to variables.
Testing
each of the two variables
(one for each
end of the tick mark) against the inductive functions an Additive function which can explain them. To produce Figure
7, however,
we find
the user needs to add four
more tick marks. The user can insert two tick marks after the left-most and two after the center tick mark, The
Figure 10: Bounded insertion Having
inserted a sufficient
number of tick marks, the user
can adjust the ends to carefully
position
adjust all of the tick marks consistently
them.
This will
using the regularity
discovered will
and represented
allow
by the inductive
the user to more flexibly
group of tick marks in a more uniform grouping
group.
manipulate
This
the entire
drawing groups
Figure groups have been implemented
user
interface
package. is shown
tool-kit The
lab for
a wide
however,
the
application
in Figure
shown that inductive
and
in C++ using the parametric
supporting
11. Our
spaces inductive
implementation
groups can be used effectively
range
has in our
of drawing
problems.
Our focus,
has been on the induction
discovery
architecture
and algorithms, not on usability experiments. been no user testing beyond our own lab.
There
has
1 shows five
interrelated. will
not support
aligned
with
Lin,
J. “A
Parametric
Coordinate
D.,
Metamouse”
Moran,
Witten
Based
University,
I.,
Graphical
as
1995.
Computer
Thesis,
Provo,
Kittlitz
K.
Procedures:
Human-Computer
T. P., Chiu,
G. “Implicit
a Freeform
[Myers92]
and The Vol.
Interaction,
Systems
P., van Melle,
Interaction {CHI
Paradigm”
‘95),
and
IEEE
Interaction
Computer
Vol. 7, No. 9, 1987.
[NIC] http:
//issl
.CS .byu.
Factors
Techniques
Graphics
edu/home.
Olsen, D. R., Ahlstrom, Geometry-based Widgets Systems
Human
in
1995.
Myers, B.A. “Creating
in Computing
W.,
Structures for Pen-Based Systems
Applications,
[Olsen95] “Building
199
Systems,
“Inferring
Demonstration”
Factors
Repetitive
Programming
in Computing
Nested
Maulsby V.,
I Do:
1, 1992.
Computing
member objects. This manipulation
non-
D. C. and Olsen, D. R. “Pictures Factors
Young
Within
means of manipulating
requires
Chapter 9, The MIT Press, 1993.
Brigham
[Moran95]
mechanism for discovering regularity in groups of objects and then allowing users to work with that regularity by
but
marks
do not handle.
Programming
Science Department, UT. 1993.
Kurtenbach,
is a general
This
What
Master’s
7, No.
result
Watch
System”,
Complete
The
“Eager:
Interface
[Maulsby92]
user interface.
fashion,
the tick
which our algorithms
Kohlert,
Franceschin
any
as keeping
the days of the week.
[Cypher93] Cypher, A. Tasks by Demonstration”
[Lin93]
in
will allow the
REFERENCES
Interactive
applied
than those
that are obviously
in a coordinated
such things
hierarchic relationships
[Kohlert95]
algorithms are generalizable to functions, Using data models by NIC, these algorithms can be
groups
groups techniques
items in the groups to work
Input Data”, Human
Figure 11: Inductive Groups Application
inductive
Our inductive
by Demonstration,
The induction discovery any number of induction similar to the one provided
techniques
object.
One facility that this system does not support is interrelationships among inductive groups. The bar chart in
SUMMARY IWC
user interface
an individual
fashion than simple
mechanisms would allow.
Inductive
requires no additional for manipulating
(CHI
by and
html
B., and Kohlert, D. Human by Example” ‘95),
1995.
[Shipman95] Shipman, F. M., Marshall, C, C., and Moran, T. P. “Finding and Using Implicit Structure in HumanOrganized in Computing
Spatial Layouts Systems
(CHI
of Information” ‘95),
1995
Human
Factors
Groups
Dan R. Olsen Jr. Xinyu
Deng
Computer Science Department Brigham Young University Provo, UT 84602 [email protected],edu
ABSTRACT The notion of inductive for
manipulating
behavior
groups is presented as a mechanism
sets of related
objects.
The interactive
for
discovering
general. An application
inductive
relationships
The work complements free form
INTRODUCTION The notion of a group
software,
inductive
form of such inductive
is found in Peridot’s handling
discovery
of lists [Myers92].
in
using these techniques is shown.
KEYWORDS: user interface interaction by demonstration.
of the group. Based on these relationships, behaviors which leverage across the entire group
can be defined. A limited
of such groups is discussed along with extensible
algorithms
definition interactive
groups,
has long been a part of drawing
graphics
the work on discovering [Moran95,
Shipman95].
structure in Their
work
attempts to discover groups and relationships among groups. In this work we take an already recognized or specified group and discover regularities in the group. Based on these regularities we provide more amplified interactions
with the group itself.
packages. The set of things that can be done with a group of objects and the ways in which those objects interrelate been quite operations scaling.
limited. allowed
Any
other
In most drawing
packages
on the entire group are translation operations
require
that the group
broken apart and the objects manipulated exception groups.
to this is Aldus Intellidraw’s
We are interested in more flexible because they across a whole
allow
independently.
Metamouse [Maulsby92] the interactive behavior actions. This produces
and be An
notion of distributed
representations
user manipulations
set of objects,
has
the only
for working on widgets
with groups by example stems by
example
pictures as input data [Kohlert95]. we only had learning finite set of relationships.
[Olsen95]
and
In both of these systems
mechanisms
which
could
handle
a
Having done that work we wanted
of groups
to be leveraged
or Eager [Cypher93] which watch of the user and infer repetitive which
our work
to extend those mechanisms to lists of arbitrary length. The inductive groups presented here are a first step towards that goal.
There are systems such as
procedures
Our motivation from
can generate or
BEHAVIOR
OF INDUCTIVE
GROUPS
Before discussing the implementation is useful to consider their behavior.
of inductive
groups it
Take for example
bar chart shown in Figure 1. There are five inductive in this chart. They are the bars themselves,
the
groups
each set of tick
modify groups of objects. Such techniques, however, do not preserve the discovered relationships between objects and
marks on the two axes, the numbers on the left side and the
must be redemonstrated
these groups carefully one can see constant properties that all members of the group share, inductive properties (where the property for the i-th object can be computed from the property of the (i- l)th object) and variable properties that have no relationship at all to the other objects.
with each new activity.
In this paper we discuss the notion an inductive group, relationships discovered
by
example
of inductive groups. In between objects are
and maintained
as part
of
the
Permission to make digital/hard copies of all or part of this material for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication and its date appear, and notice is given that copyright is by permission of the ACM, Inc. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires specific permission and/or fee. UIST ’96 Seattle Washington USA @ 1996 ACM 0-89791-798-7/96/11 ..$3.50
days of the week across the bottom.
Let us assume for our discussion
By considering
that the black
each of
bars are
rectangles which have as basic properties fill color, width, height, and lower left point. If we consider these bars as a group we see that fill color, width and lower Y coordinate are constant across all of the rectangles in the group. The X position of each bar can be determined inductively by adding a constant to the X position if the preceding bar. The
193
height
of each bar is completely
discoverable
relationship
variable.
There
is no
among the heights.
Changing a constant propetiy Suppose that the user changed the color of one of the bars in our example. The color is a constant property
The bars demonstrate An inductive all
object
computed properties properties
group can have constant properties
of the group,
inductive
properties
some function
for computing
Changing the color of one bar will change the color of all. If
shared by
the user was to grab the lower end point of one of the tick marks in Figure 2a and drag it downward, this changes the
which
a value from preceding
sequences can be characterized
value, induction function. The remaining
function
and
the
are
values.
by the initial
coefficients
of
that
Y position which is a constant property of this inductive group. The result is to lengthen all of the tick marks as shown in Figure 2b. Changing a constant property of any object changes that property for all objects in the inductive group. Note that the issue of which properties determined
groups in our bar chart further
of inductive
The
groups.
induction
because it recognizes incremented interesting.
by
illustrate
The horizontal
on the
string
row
values
the
days
of
the
themselves
follow
week
and are are more
constant or inductive an inductive
L II
al
of tick
sequence
through the names of the days of the week.
‘k‘
b
I
Figure 2: Changing constant properties
is different
that the strings are numeric
10. The
Not only is the geometry
but the strings
are constant is
at the time the group is created. This creation
process is discussed later in the paper.
marks have all of their properties constant except their X coordinates which are inductive by adding a constant. The numbers on the left side have a constant X coordinate and font, an inductive Y coordinate and an inductive string value.
of the group.
group.
by some inductive function and variable which are unique to each object. The inductive have their values defined by an initial value and
Such inductive
concept
what we mean by an inductive
Changing inductive properties If the user changes an object property induction,
which
is defined
there are two special cases to consider,
value and some induced
by
the base
value. Suppose that the left-most
tick mark in Figure 3a was moved to the right. This would change its X coordinate which is an inductive value. The left most tick mark, however, is the base or initial object for this inductive group. Moving the left-most tick mark would change the initial value of the induction. In the case of this
30
example,
20
right
right-most shown
10
coefficient
M3ncky Figure
T ues day 1:
Wednesduy
Bar Chart
Interactive behavior Having recognized and created inductive
groups, there are a
variety of interactive behaviors that can be defined on such groups. All of these behaviors are defined in terms of manipulating a single element of the group. manipulating a single element a user can leverage behavior across the entire group.
By that
When a user manipulates an object on the screen, the application is changing some of the model properties of that object. If one drags one of the bars, the X and Y of the lower left corner are changed. The impact of such changes on the entire group is different for constant, inductive variable properties in the group.
~nd
194
set of tick marks would
move to the
that the last or
tick mark in the group was moved to the right, as
in 3c. The X coordinate
induced right
the entire
as shown in 3b. Suppose, however,
value.
The
behavior
of the induction
of this tick then
is
to
mark change
is an the
so that the induced value of this
most tick mark matches the value that the user has
requested for this object. In this case the other tick marks would be repositioned to reflect the new coefficient as shown in Figure 3d. Note that changing any other of the tick marks would have a similar effect. The user would change the one that was most meaningful to the task at hand, with the induction function working out what needs to be done to accommodate
the user’s request.
Similarly in Figure 1, if the user changed Monday to Sunday, then the sequence would change to (Sunday, Monday, Tuesday) because Monday is the base value for this inductive group. If instead the user changed Tuesday to Wednesday, then the sequence would change to (Monday, Wednesday, Friday) because the induction coefficient would change to 2 days instead of one. If the user were to change the number 30 to 50 then the sequence of numbers would change to (10, 30, 50). The coefficient of 10 between each number is changed to 20 which is what is needed to
accommodate
the user’s request that the third item be 50.
This behavior changes
to
flexibility
of inductive individual
properties
objects
in the way
in response to user
provides
a whole
that users can manipulate
new
sets of
calculate
the inductive
inductive
functions
and coefficients
of the
properties.
Any properties which cannot as constants or by induction are retained
explained
be as
variable properties.
objects. Brief NIC introduction Our implementation of inductive groups was done in the Nucleus for Interactive Computing or NIC system [NIC]. There are a few aspects of the NIC architecture
which
are
important to understanding how inductive groups are implemented. The key issue here is the way in which NIC models interactive
data. Similar
techniques
can be used in
other systems.
Figure 3: Changing
Inductive Properties
Changing variable propeflies If the user was to change the height of the middle bar alone would
be changed.
of
bar, that
following
the deleted
object,
their position
are
already
properties properties
is added into the group,
defined
by
the
group
are
because
important
because their position
in the inductive
inductive
induction
sequence
DISCOVERY Having
which
can be inserted
how
the constant
properties
of
the group
and
characters
of an object are identified
indices
support
a variable
number
of
or index of a NIC object
value or another NIC object.
The most
aspect of this for our purposes is that it is possible
discovery
and object
manipulation
algorithms
from which to form the group is created from an ordered set of
In most cases the order is quite natural to the
and does not require
any special remembering
of
the order.
behave
interactively we can now address ourselves to the actual creation of inductive groups. This is performed by example. A user selects an ordered list of objects. The first object is considered to be the base or initial object on which the group is based. This example set of objects is used to determine
values
longs,
to be performed. We do not believe that this ordering problem is a usability issue once users understand that order
and
problem groups
floats,
objects. A user uses standard selection mechanisms such as “shift-click” for selecting multiple objects one at a time. The order in which the objects are selected is important because that forms the sequence across which induction is
GROUPS inductive
are primitive
Objects have a class, zero or more attributes and
is important.
OF INDUCTIVE
discussed
elements
there
values include
Sequence of objects An inductive group
object copies the variable properties of the object which precedes it in the group. The user is then free to change them as desired. This addition and deletion feature allows inductive groups to take on the behavior of semi-uniform into
data model
Primitive
which can apply to any object in the user interface.
has changed. The variable properties of the new object are undefined. As a convention in our implementation the new
lists of objects deleted.
data
at run-time to identify and retrieve all of the component values of an object. This makes it possible to write general
can be computed from the group. The inductive of all objects following the new object must be
recomputed
manipulating
model and an extensible set of induction functions. Thus the mechanisms are the same for lines, rectangles, days of the
objects that support
of the object the
and
in terms of a general
them. The value of any attribute
it is added after
and
discovering
in a wide variety
packages. For this reason all
discovering
are defined
can be a primitive
object. The constant properties
that we define for
by a symbolic name and indices by an index varying from zero to the number of indexed values in the object. Most
properties remain the same.
some existing
for
zero or more indices. Attributes
in the sequence has changed and their variable
When an object
groups
and NULL.
group only the
are recomputed
not just drawing
mechanisms
inductive
objects.
their constant properties
properties
architecture
groups which could be applied
In the NIC
it in the group are changed. For all objects
the same, their inductive
the
it was very important
a general
week or tasks in a Pert chart.
at the time the
Adding and deleting objects When an object is deleted from an inductive objects following
and
of situations,
The heights of the bars are
was discovered
this work
algorithms inductive
variable and thus unrelated to each other. The height of each bar functions independently. The fact that bar height is variable across this group group was created.
In doing
In our example the user might first draw the three black bars as in Figure 1. Having positioned the bars as shown in the figure, the user can then select them in left to right order. This makes the left-most bar the base for the inductive sequence. Having selected the objects, the user requests that
and to
195
an inductive
group
be formed
either
using the menu
or
accelerator keys, Discovery of constants and variables Once the user has presented an ordered list of objects to be formed into an inductive group, the first differentiate between the constant and variable
step is to parts of the
objects. Selection of our example black bars would produce the list of NIC objects shown in Figure 4.
[Width
20]
[Width
20]
[Width
c
[Height
501
[Height
20]
150]
[Height
1001
induction functions are dates, months, log scales and various string inductions such as (XO1, X02, X03) or (Va, Vb, Vc). The lnductFunct abstract class The induction discovery algorithm
must be generalized
broad class of induction
In order to accommodate
functions.
to a
this, we define the abstract class InductFunct. All of the inductive group algorithms are based on this abstract class.
[Color
(O O 0)1)
[Color
(O O 0)1)
[Color
using string representation of numbers, and induction on strings which are names of days of the week. Other possible
Our implementation
of inductive
subclasses of InductFunct. registry
(O O 0)1)
groups has quite a number
These classes are placed
where the general algorithms
makes the addition
in a
can find them. This
of new forms of induction
very easy.
Figure 4: Selected list of NIC objects The InductFunct abstract class has three methods that form the primary interface to the rest of the system.
The first step is to compare all of the objects to the base, or first, object. This comparison is recursive and will identify all of the components components pattern
which
are replaced
shown
portions ofthe
are different.
by variables.
in Figure
This produces
– This method
CalcCoeff
These different
and
argument
the
[X
5. Note that all of the constant
objects have remained,
~Width
!Vl]
[Y
201
between
10]
~Heiaht
!V21
[Color
Figure 5: Pattern generalized
1
(O O 0)1)
a value
implementation
from examples
the
fuzzy
form
corresponding
is different
from
the
base.
In
a variable
is
introduced
only
values are more than a little different.
that are close to each other are considered
if
variables
the and
This
{ {
50, 50,
Figure6:
150, 150,
250 100
Sequences
how
close
the
calculated
method
takes
an index
into
the
is the
properties
or initial
value so that
have the new value way
in
which
the
specified.
changes
to
are handled.
– This takes an element index, calculates the value of that element using the stored coefficients and
BuildElement
returns that element. This is the method for regenerating the sequence from the inductive function.
from
To better understand how these methods work, consider the We can first create a sequence of values for !Vl. DaysOfWeek
induction
function
object.
We
can
call
CalcCoeff on the DaysOfWeek object passing in the sequence { 50, 150, 250 }. The result will be 0.0 because DaysOfWeek expects a sequence of day names. We next try an Additive induction function object. Its CalcCoeff method
6. These values are
} }
This
value will
method
inductive
Inthepattern generated in Figure5 there are two variables, !Vl and !V2. Each of these variables ~e associated With the
: V2
–
the indexed
which are truly variable.
!Vl
1.0 for
This will change the coefficients
constants. Our next step is to determine which of the variables can be described by an induction function and
sequences of values shown in Figure drawn from the example objects.
and
sequence and a new value for the element at that index.
the base value is retained as a constant. This fuzzy comparison allows for the fact that users cannot draw their examples exactly. Discovery of induction variables The previous step has differentiated
For values that are close, it returns a result 0.0
ChangeCoe#
In
Values
equivalent
induction
coefficients are stored in the induction function object. This object can now be used in place of the original example list of values.
our
we used a fuzzy form of this comparison.
calculate
induction is to the example values. This method is the heart of the induction discovery. The calculated
This generalization from examples to a pattern with variables has two forms. The crisp form creates a variable whenever
to
coefficients which can generate that sequence. This method returns a 1.0 if it was exactly successful and 0.0 if it failed.
(:Rect
takes a sequence of values as an
attempts
returns 1.0 and stores the initial value 50 and coefficient 100 in the function object. The additive induction function
I
exactly explains the sequence of numbers. If the sequence for !V2 { 50, 150, 100 } were passed to an Additive induction function it would probably store 50 as its initial value and 25 as its coefficient but would return 0.2 or less
ofvalues
There are many possible induction functions that could be used, In our example we use an additive function, additive
196
as its result because the induction
generalize foreach
does not explain the value
very well.
(I*coe~
method actually
For example + init)
the Additive
calculates the induction induction
function
HF
= null
HC
=
is
where I is the index of the element. Given
return 50, 150 and 250 respectively.
The
method
properties.
most bar in Figure position
P
This reconstructs
supports
the
changing
of
?
Suppose that the user moves the right-
function
explained
by
HF
}
object that is
as the new value. The Additive function object would recalculate the coefficient to be 105 instead of 100. By
}
HC > threshold then V is an induction property else V is truly variable
the bar at 260 instead of 250. The system would on the Additive
F
if
1 to the right by 10 pixels. This would
invoke ChangeCoeff
sequence S for from the examples of induction function
create an induction function object NC = F. CalcCoeff (S) note that F now contains the ca lculated coef ficients if NC ~ IiC then { HF=F HC = NC
the
describing the X position of the bars. ChangeCoeff would receive 2 as the index of the element being changed and 260
V
L
sequence.
ChangeCoeff
inductive
pattern
0.0
compute value variable foreach class r
the induction function discovered above. Calling BuildElement on that object with indices of O, 1 and 2 will inductive
to form in P
{
The BuildElement function.
examples variable V
It is important
to note that although
the general matching
algorithm is recursive, the check for inductive is not. The cost of this check is
calling BuildElement we can reconstruct the new sequence as 50, 155 and 260. These reconstructed values can be used
NumberOfFttncs
relationships
* NumberOfVars
to redraw the bars in their new locations.
Representing
In the case of the DaysOfWeek
The information about constants, variables and inductive variables that is discovered by these algorithms is assembled
process induction
is
similar
to
that
class, CalcCoeff
subclass of InductFunct, described
for
the
the
Additive
will attempt to find a sequence of
strings that are ordered by days of the week and will store a start day and a days-to-increment as its coefficient. BuildElement will use a modulo function and a table lookup to return the day name for elements in the learned sequence. ChangeCoeff can be used to change the starting day or the increment to be used. These three methods provide for a wide variety of inductive
behaviors to be represented.
together in an InductiveGroup object supports interaction the interprets
that will
for each variable.
explain
the sequence of example
This discovery
possible to create several inductive
function
groups and then group
group.
SPACES
situations
the relationships
appear to be quite complex.
Consider
among
objects
for example
can
the tick
marks around the edge of a dial shown in Figure 7.
values
creating an object method using the
example values. The function which returns the highest confidence is retained and if that confidence is above the inductive
objects and
as changes to the inductive
process consists of taking
each registered subclass of InductFunct, of that class and trying the CalcCoeff
threshold
that interaction
them as a larger inductive PARAMETRIC
Induction discovery algorithm The induction discovery algorithm consists of discovering the variables and then trying to discover an induction
object. The InductiveGroup
with the constituent
description of the group. The inductive group that is assembled using these algorithms its itself a NIC object. It is
In many
function
the group
I
object is stored in place of
I
the sequence of example values. If more than one function is above threshold, then the function that returned the
Figure 7: A dial
highest vah,te is chosen. If there is more than one function with that same highest value, then the first one in the list is
The tick marks are evenly spaced around the edge of the dial, but in Cartesian coordinates their inductive relationship is sinusoidal, requiring several coefficients and thus 5 or
chosen. The implementor can order this list to select the most likely functions first. The full algorithm is as follows.
more examples to learn the inductive
function
correctly.
As in our work on widgets by example [Olsen95] we simplify the problem using parametric spaces. In our drawing tool, objects are drawn on top of other objects and use the coordinate systems defined by those objects. The
197
background
has a standard rectangular
The base of the example coordinate
system.
dial,
The tick
coordinate
being
marks,
system of the arc can be explained
system.
an arc, has a polar using
problem,
however,
is that after inserting
the first tick mark,
the result is as shown in Figure 9.
the coordinate
by the same inductive
functions used for the tick marks of the bar chart which are in rectangular coordinates. Our drawing relative
to
system defines parametric rectangles
(normal
Cartesian), parallelograms (polar) and splines (“curvy” offered
objects
will
lines
[Lin93] (angled
(oblique coordinates), arcs coordinates). The affordance
to the user by this model
‘circular’
coordinates
Cartesian),
function
is that “drawing
in a ‘circular’
way
on and
drawing on ‘curvy’ objects will function in a ‘curvy’ way”. We believe (without experimentation) that this is a naturally understood
mechanism
for
expressing
relationships.
\ Figure 9: After tick mark insertion
The
result is that the induction discovery process is greatly simplified and the number of examples required is reduced.
Inserting
the tick mark moved all of the others around. The
user can grab the stray tick mark and drag it back to the ending INTERACTION Having
WITH INDUCTIVE
defined
the
groups we would
algorithms
like to revisit
and show how the machinery take for example Figure 7.
GROUPS for
managing
their interactive
accomplishes
the creation
position
where it belongs.
Dragging
mark back will cause a ChangeCoeff inductive
recalculate
behavior
The user can therefore
the goal. Let us
of the dial
tick
marks
in
the coefficients
and reposition
the tick marks.
insert the desired
number
of tick
marks and then drag the last one to its desired location. The result is the evenly spaced set of tick marks in Figure 7. The example
of the dial tick marks is a form
Suppose that first our user drew three tick marks as shown
insertion
in Figure
extend indefinitely
8. Because they are drawn on an arc they are in
the stray tick
method to be called to
of bounded
where the user does not want the list of objects to but rather stay within
the bounds of the
polar coordinates (a,r) where a is the angle and r is the radial distance from the center of the arc. The user can then
arc. To accommodate this we have provided explicitly for bounded, as opposed to free, insertions. When a bounded
select these lines
insertion
in left
to right
order
and request
the
formation of an inductive group. The color and line width of the lines are the same and will become constant properties of the group.
is performed,
we first copy the values of the last
the object in the induction sequence. After performing insertion (or deletion), we use ChangeCoeff to put the values of the last object back to what they were before the insertion. This has the effect of locking first object coefficients problem
both the last and the
in place and then adjusting appropriately. Using bounded
shown
in Figure
9 would
Instead after the first insertion, as in Figure 10.
not
the induction insertion, the have
the result would
occurred. have been
Figure 8: Example dial tick marks
Looking carefully at these tick marks we see that their end points all share the same r coordinate. These too are constant properties. The a coordinates of the end points, however, are all different and will be generalized to variables.
Testing
each of the two variables
(one for each
end of the tick mark) against the inductive functions an Additive function which can explain them. To produce Figure
7, however,
we find
the user needs to add four
more tick marks. The user can insert two tick marks after the left-most and two after the center tick mark, The
Figure 10: Bounded insertion Having
inserted a sufficient
number of tick marks, the user
can adjust the ends to carefully
position
adjust all of the tick marks consistently
them.
This will
using the regularity
discovered will
and represented
allow
by the inductive
the user to more flexibly
group of tick marks in a more uniform grouping
group.
manipulate
This
the entire
drawing groups
Figure groups have been implemented
user
interface
package. is shown
tool-kit The
lab for
a wide
however,
the
application
in Figure
shown that inductive
and
in C++ using the parametric
supporting
11. Our
spaces inductive
implementation
groups can be used effectively
range
has in our
of drawing
problems.
Our focus,
has been on the induction
discovery
architecture
and algorithms, not on usability experiments. been no user testing beyond our own lab.
There
has
1 shows five
interrelated. will
not support
aligned
with
Lin,
J. “A
Parametric
Coordinate
D.,
Metamouse”
Moran,
Witten
Based
University,
I.,
Graphical
as
1995.
Computer
Thesis,
Provo,
Kittlitz
K.
Procedures:
Human-Computer
T. P., Chiu,
G. “Implicit
a Freeform
[Myers92]
and The Vol.
Interaction,
Systems
P., van Melle,
Interaction {CHI
Paradigm”
‘95),
and
IEEE
Interaction
Computer
Vol. 7, No. 9, 1987.
[NIC] http:
//issl
.CS .byu.
Factors
Techniques
Graphics
edu/home.
Olsen, D. R., Ahlstrom, Geometry-based Widgets Systems
Human
in
1995.
Myers, B.A. “Creating
in Computing
W.,
Structures for Pen-Based Systems
Applications,
[Olsen95] “Building
199
Systems,
“Inferring
Demonstration”
Factors
Repetitive
Programming
in Computing
Nested
Maulsby V.,
I Do:
1, 1992.
Computing
member objects. This manipulation
non-
D. C. and Olsen, D. R. “Pictures Factors
Young
Within
means of manipulating
requires
Chapter 9, The MIT Press, 1993.
Brigham
[Moran95]
mechanism for discovering regularity in groups of objects and then allowing users to work with that regularity by
but
marks
do not handle.
Programming
Science Department, UT. 1993.
Kurtenbach,
is a general
This
What
Master’s
7, No.
result
Watch
System”,
Complete
The
“Eager:
Interface
[Maulsby92]
user interface.
fashion,
the tick
which our algorithms
Kohlert,
Franceschin
any
as keeping
the days of the week.
[Cypher93] Cypher, A. Tasks by Demonstration”
[Lin93]
in
will allow the
REFERENCES
Interactive
applied
than those
that are obviously
in a coordinated
such things
hierarchic relationships
[Kohlert95]
algorithms are generalizable to functions, Using data models by NIC, these algorithms can be
groups
groups techniques
items in the groups to work
Input Data”, Human
Figure 11: Inductive Groups Application
inductive
Our inductive
by Demonstration,
The induction discovery any number of induction similar to the one provided
techniques
object.
One facility that this system does not support is interrelationships among inductive groups. The bar chart in
SUMMARY IWC
user interface
an individual
fashion than simple
mechanisms would allow.
Inductive
requires no additional for manipulating
(CHI
by and
html
B., and Kohlert, D. Human by Example” ‘95),
1995.
[Shipman95] Shipman, F. M., Marshall, C, C., and Moran, T. P. “Finding and Using Implicit Structure in HumanOrganized in Computing
Spatial Layouts Systems
(CHI
of Information” ‘95),
1995
Human
Factors
