J.W. van Aalst*, T.T. Carey, D.L. McKerlie
Department of Computing and Information Science
University of Guelph
Guelph, Ontario, Canada N1G 2W1
email: tcarey@snowhite.cis.uoguelph.ca
email: mckerlie@snowhite.cis.uoguelph.ca
*Delft University of Technology
Department of Technical Mathematics and Informatics
Delft, the Netherlands
email: jw@snowhite.cis.uoguelph.ca
"Teaching as mediating learning involves constructing environments which afford learning about descriptions of the world...to allow more to be learned than is available from experience alone." [12]
Learning about design is a central component in education for human-computer interaction [5]. To teach basic design skills we have to provide learners with opportunities to "learn by doing" on concrete projects, and opportunities to "learn by reflecting" [17] on both their own projects and case studies of other user interface designs. We have found Design Space Analysis (DSA) [13] to be a useful technique for students learning user interface design skills. Design Space Analysis provides a vocabulary of Issues, Options and assessment against Criteria, and a notation to record the current state of a design in shorthand form. In previous research, we have used adaptations of DSA to promote reflection on design case studies [4, 3]. In the FLUID tool described here, we have incorporated explicit instruction on design and problem exercises for learners, yielding an interactive multimedia system to be incorporated into an HCI course. In the course, the students will work in pairs with FLUID as active preparation for studio sessions involving group design projects.
In the spirit of "teaching as mediated learning" as defined above, we use DSA as a description of the considerations emerging in design. For case studies, this allows us to summarize the important design advances in ways that encourage generalization. For the design exercises, it allows us to support design activities by presenting video clips of advice and war stories. We use the "training wheels"1 analogy to clarify the role of DSA in the learning context. Here is the explanation given to users of FLUID:
"We will be asking you to construct your design according to a design method called Design Space Analysis (DSA). We use DSA here as a "training wheels" system for learning design. Remember the training wheels on your first bicycle? They helped you to get going and to avoid initial problems. Before long you found you didn't need the wheels any longer because you had internalized the necessary skills. Our requirement that you use DSA is like that: good designers may not have to be so explicit about these steps any more because the processes have become automatic (well, not quite automatic - often we still need to reflect on the design process, so you will find DSA handy later on as well)."
We thus would like to treat Design Space Analysis as a technique for representing design considerations and processes, not as a theory of design practice. However, any notation carries with it implications for the task processes it supports. One interesting aspect of the development of FLUID has been the resulting observations about Design Space Analysis and its implicit model of design.
The case studies in FLUID note that the material has been adapted to serve the training wheels role: the original designers did not necessarily represent their designs in this way. However, design documentation normally rationalizes design history to highlight key decisions [15]. The FLUID tutorial also notes that Design Space Analysis can be useful in other specific roles during design [2].
In the next section we describe a walkthrough of a session with FLUID as a typical learner might experience it. This raises interesting questions about how we learn to design and how teaching might mediate and extend the learning process. In section 3 we discuss the Design Rationale for FLUID, including links to other current research on interactive tutoring systems and reflection about our design based on usability testing. In section 4 we present our observations on Design Space Analysis as a training wheels aid for learning user interface design, including notes on what we learned about DSA through the process of specifying its use in the FLUID Tutorial. The concluding section outlines our current work to deploy and generalize the FLUID system.
FLUID is a learn-by-doing system in which the central learning task is a design exercise. Currently, the learner is asked to be the user interface designer of an interface which makes it easy to place, move and rotate furniture objects in a virtual room, using a standard mouse for direct manipulation. This exercise is adapted from a project report by Stephanie Houde of Apple Computer [9].
Initially, the learning task is presented by a team Manager character appearing in a video window. The learner also meets the client (a furniture shop manager), as well as two other characters who will offer advice during the design process - a Tutor on DSA and a senior Designer. Figure 1 shows the Manager and Tutor pointing out various aspects of FLUID that the learner may wish to explore.
Figure 1. The Tutor and the Manager, introducing the student to the project.
Typically, learners will work through the DSA Tutorial first, although they are free to move back and forth amongst the other components including a Library of worked cases, a Table of Concepts and the design Workbench in which the exercise is carried out (all discussed further below).
The Tutorial introduces the concepts of Design Rationale, DSA, and the DSA notation. In addition, the Tutorial contains some real life "war stories", identifies some disadvantages of DSA, summarizes the different roles that DSA takes on during design, and offers guidelines as to when it is (and is not) advisable to use DSA. The prototype case Library currently contains a single exemplar system, adapted to use the vocabulary and process of DSA. This case is about the design of a device for easy rotation of an object about an arbitrary axis in 3-D space [6]. For the exercise and the worked case, we deliberately chose two related topics to make it easier for the learner to apply the lessons from the case study. Figure 2 shows data from usability testing in the case study (the case study also contains a video clip of a 1981 3-D rotation device [7]).
Figure 2. A case from the exemplar Library.
The FLUID design Workbench provides the
standard tools of DSA, including a DSA diagram
to describe the design Issues (footnote3)
,
Options, and Criteria, a usage Scenario to
describe typical interactions in narrative form,
and a design history log which we can use later
to show the learner's actions. In addition to the
Tutor, Manager, and senior Designer, the learner
can also access a group of Users; they report on
the usability of the emerging design. The
comments of the users are taken from usage data
in the published report [9]. A Demonstration is
available, to take the learner step-by-step
through the basics of filling in a DSA diagram in
the Workbench. Figure 3 shows the various
navigation and guidance features.
In the introduction of the design exercise, the
Manager presents an initial idea for the interface
(corresponding to the starting point in [9]). The
initial usage Scenario repeats the Manager's
description, and the DSA diagram likewise is
seeded with this initial information. If learners
request a user test at this point, the Users4 will
tell them that more details will need to be filled
in about how the interface works. To add detail
or change the current design, learners edit the
usage Scenario and the DSA diagram. By
selecting a set of design choices, the learner can
form a new design Alternative and present it to
the Users.
The bottom left shows
the FLUID components: Table of Concepts, Tutorial,
Case library, and Design Workbench.
For example, Figure 4 shows the state of a
design after the learner has recognized one of the
limitations of the initial idea. In the initial idea
(as presented in [9]), a hand-shaped cursor is
used to directly manipulate points on the
furniture objects, each point corresponding to a
different action such as lifting, sliding and
rotating. In practice, users had divergent
expectations about which part of an object would
correspond to a particular movement. The design
team introduced "narrative handles", hands
which appeared on an object and indicated, by
their orientation, the movement invoked by
selecting them.
Figure 4. the FLUID Design Workbench.
To add this Option as a possible choice for a new
design, the learner must edit the usage Scenario
to describe a new style of interaction and then
edit the DSA diagram to record this choice
(these edits can be in either order). In Figure 4
the learner is adding the Option ‘Clickable boxes
on objects' as a new solution to the Issue called
‘What focus-object metaphor to use?'.
Throughout this process, the Tutor, Manager and
senior Designer provide help on request. For
example, if the learner requests assistance after
hearing from the Users that "the hands
don't seem to be in places they expected",
the Tutor will suggest (in a video clip):
"Think of some other ways to do things (a
new Option)".
In order to provide this context-sensitive
assistance, FLUID must monitor the learner's
actions and map them against a state chart which
determines what advice to offer. This requires
the following features during the process of
changing a usage Scenario (footnote5)
or DSA
diagram:
We discuss in section 3 the rationale and effects
of these features on the learning process.
At the end of the design exercise, the learner can
review the project with the various characters. If
the design is incomplete, the characters suggest a
return to the design Workbench, otherwise they
congratulate the learners upon their exercise and
suggest follow up reading before the scheduled
group design studio exercise.
FLUID was tested with prospective students of
the University of Guelph undergraduate HCI
course. Completing their DSA diagrams using
the design Workbench took on average 59
minutes with a range from 39 to 90 minutes.
This does not include the time taken to work
through the Tutorial or Library case.
We observed that FLUID users understood the
context of their mission in terms of the cover
story and got into their roles. However, we need
to reinforce the cover story when they encounter
the Workbench, since for typical students, the
Tutorial and case Library are used after the
mission description and before the Workbench.
Figure 5. Quadrant of the Table of Concepts.
The Tutorial supports learners who prefer to
observe and reflect before launching into a task
on their own, and the case Library supports
learning by example. In the HCI course, this
structure also allows us to relate FLUID to other
content: students later do a learning styles
assessment and discuss its implications for
supporting users with various learning styles.
The Tutorial and case Library appeared to
provide users with the background information
necessary for learning about DSA. All users
explored both interaction elements, suggesting
that perhaps these provide complimentary
information rather than redundant information.
No users referred back to these interaction
elements during construction of their DSA in the
Workbench.
We also wanted the design exercise to be novel
for the learners, both for motivation and to avoid
routine copying from an existing system. Finally,
the exercise had to be non-trivial but not require
extensive domain knowledge.
The published case study by Stephanie Houde
[9] met all of these criteria. A scanned image of
the paper is available in FLUID after learners
complete the design exercise. The choice of
another study to include as a worked case
example was largely determined by this choice
of design exercise. The study by Chen et al. [6]
addressed a similar design problem but with
different constraints (i.e., designing a new
controller device in [6] versus designing a new
way to use an existing device in [9], for similar
tasks).
We found that users appreciated the concrete
examples provided by both cases. However,
there was no evidence that selecting cases from
the same domain was advantageous, suggesting
that our concern over similarity between cases
may have been exaggerated.
Through pilot testing we established the likely
choices, and then structured FLUID to contain
nested lists which expose further choices from
‘Other' submenus. The final ‘Other' submenu
notes that the user has produced a choice not
included in FLUID and requests feedback by
email to the FLUID developers.
The successful aspect of limiting design Options
was that users could eventually build an
understanding of the set of Options and
formulate a meaningful Issue for the DSA
diagram. Issues were not constrained in any way,
allowing users to create their own labels for each
Issue. The well-constructed set of Options
provided an opportunity to guide learning about
Issues - from previous work, Issues are known
to be a difficult concept to teach.
However, test users commented that they felt
constrained by these anticipated design Options.
With our initial test users, unanticipated Options
generated during usage testing (e.g., a 3-D crane
instead of the 2-D hands) could have been
allowed without jeopardizing the rest of the
design space.
There was great variability in use of characters'
advice between users during construction of the
DSA diagrams. Some users consistently asked
for advice from the characters after each addition
to the DSA diagram, while others only sought
approval after their DSA diagrams were
complete.
We observed that users recognized the
differences in the type of advice offered by each
of the characters. In general, users preferred the
advice from the DSA Tutor for two reasons: the
content of the advice seemed most applicable to
their task, and the videos for this character
tended to be the most engaging. Users
consistently commented on the entertainment
value of the video clips. These also appeared to
be an important motivator for learners to
continue to discover new information. However,
we noted that in some cases the entertainment
value may have been "too engaging" (see also
[10]). In some instances the media novelty
caused users to miss the important point in the
content.
"If all your Options are unsatisfactory on
a particular usability Criterion, try to identify a
new Issue which groups these Options together
and suggests a new design dimension to
consider."
We also noted the assumption in DSA that an
assessment on a specific Criterion can always be
related back to the choice of a design Option.
This assumption is suitable for the training
wheels setting but oversimplifies the design
space. For example, in [9] the Option of a hand-
shaped cursor satisfies the Criteria of being
natural and guessable initially. But once the
Option of hands appearing on an object or box is
chosen, the assessment of the cursor Options
changes. Thus the assessments must be
considered on the Alternative system design
rather than just the individual Options. The
closing section of FLUID uses this example to
illustrate how the training wheels experience
needs to be extended to cope with more complex
situations.
"DSA is a good tool for developing an
outline of systematic Options and solutions for
most scenarios".
In a pretest questionnaire, we asked test users to
describe a previous interface design experience
and the decision they made. After the session
with FLUID, most were able to suggest ways
DSA could have helped them during their design
process.
The FLUID system raises questions about what
it is that experienced designers know and how
others might come to acquire that knowledge
most efficiently. We designed FLUID to address
particular difficulties which we knew our
students encountered as novice designers, in the
context of a typical computer science program.
The knowledge we gained about DSA in the
process of building FLUID was an unanticipated
but welcome outcome.
We will be using FLUID in HCI classes in
Guelph and Delft in the coming year, probably
with pairs of students working together. Further
design enhancements are anticipated as we learn
more about teaching DSA in this way and more
about the usability of FLUID. We have also
agreed to distribute versions of the software to
other educational institutions and some industrial
groups, who agree to furnishing in return both
their feedback and additional worked cases and
exercises. In this way we hope to accumulate a
larger body of exemplar cases. At some point
this collection should reach a volume which
requires support for learner access to the cases
[11], but we do not envision needing this in the
short term.
We are also generalizing the FLUID architecture
to permit application to additional domains, both
within and beyond user interface design. Within
user interface design, the next skill to be taught
is the design of an evaluation. Outside user
interface design, we will be constructing
instructional modules for the tasks of problem-
based writing and of the role of a field biologist
in the Arctic. The key element here is to allow
users to engage with the design domain in a way
complex enough to seem natural but simple
enough to permit monitoring by the environment
so that feedback and support can be context-
sensitive.
3. DESIGN RATIONALE FOR FLUID
We outline in this section a number of design
issues raised by FLUID, and evaluate the choices
we made based on preliminary usability testing.
Objectives
Novice user interface designers have to develop
expertise in the following basic design skills:
The first and last of these objectives are not
addressed in the current version of FLUID. Our
use of DSA highlights the other objectives for
learners, because DSA requires explicit steps
corresponding to each of those skills and the
creation of an explicit artifact of DSA as a result.
This mapping allows us to offer focused advice
based on the DSA constructed at any point.
FLUID Conceptual Model
We based FLUID on the Goal Based Scenario
concept of Schank and his colleagues [16, 1].
This approach combines principles of active
learning, case-based teaching and expert
tutoring. In the Goal Based Scenario
architecture, learners experience the learning
task through a cover story (e.g., you
are a designer and your manager is giving you an
assignment), a mission (e.g., design
a way to manipulate the furniture in 3-D with a
standard mouse) and a focus for
actually doing the task (e.g., our Design
Workbench).
FLUID Interaction Style
We incorporated in FLUID interaction elements
to provide flexibility in learning styles. A Table
of Concepts, still under construction, supports
learners who want to develop an explicit
conceptual model of the knowledge domain
(learners can currently access a concept map of
usability criteria, containing definitions and
examples from published usability studies.
Figure 5 shows one quadrant of the concept
map).
Choice of Design Exercise
For the exercise which learners would work on,
we wanted to adapt a published case study from
an industrial environment, so that:
Design Workbench
We are not satisfied with the separation between
the design Workbench, where learners build a
DSA for the exercise, and the other products of
design such as the sketches they might make as
they work out their designs. The Workbench
manipulates DSA, not the entire design. We
knew we would not be able to build a design
toolkit and also integrate the instructional
component, so we accepted the compromise of
just developing the DSA Workbench and leaving
learners to construct other design products in
some other medium (the other activities learners
should be doing in parallel with FLUID are
described by the senior Designer in the Tutorial).
We discuss in section 5 below some of the issues
involved in creating a more capable Workbench.
None of the users expressed a need for more
creative tools to express their design ideas,
although one user took great effort to describe
his design idea to the observers of his session
with hand gestures. Other users made
suggestions for improvement.
Providing Assistance
"For maximum learning to take place, the
student should receive just enough of a push or
guidance to get past the sticking point without
being explicitly told the answer" [8].
It is possible to talk of a future system in which
learner input is analyzed for natural content and
an appropriate interpretation is made of the
meaning of a design choice. There is no such
intelligence in the current FLUID system. When
users describe a new design Option, they are
prompted to identify their choice from a list of
preset labels. This mechanism will fail if the list
of available choices is not complete enough -
i.e., FLUID does not anticipate the users' choices
- or if the list presents choices the user has not
yet thought of and thereby pre-empts some of the
desired learning.
Roles for Advisor Characters
Ideally, each of the characters providing input in
the design exercise should have a distinctive
role, so that learners know who to ask for
specific needs, and to provide context-sensitive
help [8]. We tried to structure the on-line
characters so that the DSA Tutor and project
Manager provided generic help about the design
process, and the Senior Designer provided
specific help about the design domain (and
occasional reflections on the role of DSA outside
the training wheels setting). This proved to also
be a useful distinction in terms of future
generalizations of the system: for a new design
exercise, the specific hints from the Designer
would need to be changed but the Tutor's and
Manager's video segments are re-usable as is.
4. DSA AS TRAINING WHEELS
FOR USER INTERFACE DESIGN
Issues as Design Dimensions
We found it necessary to emphasize the concept
of design Issues in DSA as characterizing
dimensions along which designers could look for
alternatives. Making this explicit in the Tutorial
helped clarify for learners the nature of an Issue
in DSA. For example, at a key point in the
design exercise, the learners must recognize that
their conception of an Issue is too limited: seeing
the Issue as "how are users shown selectable
parts of an object?" precludes the idea of a
manipulable box outside the object itself, which
requires the more general conception of the Issue
as "how are users shown how to grab the
object?".
Role of Scenarios
We had previously found that usage Scenarios
were a valuable addition to DSA [4]. Initially,
the assumption was made for FLUID that we
could easily map changes to the usage Scenarios
into changes in the DSA diagram; in the initial
FLUID versions this was the only way to change
the DSA, to encourage learners to think in
Scenario terms. Our assumption proved incorrect
- sometimes a new design Option required
several changes in the usage Scenario,
sometimes a single change in the usage Scenario
required alteration of several design Options. We
now recognize that the usage Scenarios present a
complementary view to the other DSA
components. In the current FLUID, users can
edit either the Scenario or the DSA diagram.
FLUID detects most but not all of the required
changes in the other component.
Alternatives Versus Options
We wanted to include usability testing as one of
the steps in which learners would engage.
Initially, we tried having learners select a set of
Options to present to the users, representing their
choices on the various Issues. We found this
approach was too abstract, and failed to convey
that the design becomes more than just the sum
of its parts (see also the point on decomposition
below). The explicit notion of an Alternative
system design was introduced to fill this gap.
Users were observed to struggle with the notion
of an Alternative as a set of selected Options
(one from each Issue). This may reflect a
weakness in the Case Study and the Tutorial.
This will be explored further with students in the
HCI design course.
Implications of DSA
Notation on Design Processes
As outlined earlier, we found that having to
specify the advice to be given to learners in each
process step forced us to analyze DSA as a
theory of design. We had to develop general
heuristics to guide the Tutor's video advice,
independent of the particular exercise. For
example, in one instance, the tutor says:
5. CONCLUSIONS AND FUTURE WORK
We found that most learners who were
introduced to DSA through FLUID were able
afterwards to describe in their own words what
DSA is. For example, one user said:
Acknowledgments
We thank Michael Chen and Stephanie Houde
for permission to adapt their work and their
comments on the results. Helpful comments
were also made by Allan MacLean, Jenny
Preece, Richard Jacques, Jonathan Swallow and
Charles van der Mast. We would also like to
thank the character actors in FLUID who helped
bring it to life. This work was supported by the
Natural Sciences and Engineering Research
Council of Canada, and the University of
Guelph's Instructional Development Grants
program. We thank the copyright holders who
allowed us to incorporate their material - ACM
for [9], and the National Research Council of
Canada for the videotape related to [7].
References
1. Bell, B.L. and R. Bareiss, (1993), Sickle
Cell Counselor: Using a Goal-Based
Scenario to Motivate Exploration of
Knowledge in a Museum Context, in
Proceedings of AI-ED 93, Association for
the Advancement of Computing Education,
153-160.
2. Carey, T.T., A. MacLean, and D. L.
McKerlie, Event-based Retrospective
Design Rationales, in preparation.
3. Carey, T.T., M. Ellis and M. Rusli, (1993),
Re-using User Interface Designs:
Experiments with a Prototype Tool and
High-level Representations, in R. Winder
(ed.), Proceedings HCI'94 - People and
Computers VIII, Cambridge University
Press.
4. Carey, T.T., D. McKerlie, W. Bubie and J.
Wilson, (1991) Communicating Human
Factors Expertise Through Usability Design
Rationales, in D. Diaper and N. Hammond
(eds.), Proceedings HCI'91 - People and
Computers VI, Cambridge University Press,
117-130.
5. Catrambone, R., and Carroll, J.M., (1987),
Learning a Word Processing System with
Guided Exploration and Training Wheels, in
J.M. Carroll and P.P. Tanner (eds.),
Proceedings of CHI+GI'87 Human Factors
in Computing Systems and Graphics
Interface, New York: ACM,. 169-174.
6. Chen, M., Mountford, S.J., Sellen, A.,
(1988), A Study in Interactive 3-D Rotation
Using 2-D Control Devices, in Computer
Graphics, 22(4), ACM, 121-129.
7. Evans, K.B., Tanner, P.P., Wein, M.,
(1981), Tablet Based Valuators that Provide
One, Two, or Three Degrees of Freedom,
Prceedings of SIGGRAPH'81 Computer
Graphics 15(3), 91-97.
8. Feifer, R. and R. Holyoak, (1994), Helping
Students Get Unstuck, in proceedings ED-
Media 94, Association for the Advancement
of Computing in Education, p. 645.
9. Houde, S., (1992), Iterative Design of an
Interface for Easy 3-D Direct Manipulation,
Proceedings of CHI'92, ACM, 135-142.
10. Jacques, R., (1994) Engagement in the
Design of Educational Hypermedia, Ph.D.
thesis, Southbank University, U.K., in
preparation, 1994.
11. Kolodner, J., (1993), Case-Based
Reasoning, Morgan Kaufman.
12. Laurillard, D. (1993), Rethinking University
Teaching, London: Routledge p. 26.
13. MacLean, A., Young, R., Bellotti, V., and
Moran, T., (1991), Design Space Analysis:
Bridging from Theory to Practice via Design
Rationale, in Proceedings of Esprit'91,
Office for Official Publications of the
European Communities, Luxembourg, 720-
730.
14. Moran, T., (1994), quoted in Interview with
Tom Moran, in J. Preece et al., Human-
Computer Interaction, Addison-Wesley, p.
349.
15. Parnas, D.L., Clements, P.C., (1986), A
Rational Design Process: How and Why to
Fake it, in IEEE Transactions on Software
Engineering, 12(2).
16. Schank, R.C., (1994), Active Learning
through Multimedia, IEEE Multimedia,
1(1), 69-78.
17. Schoen, D.A., (1987) Educating the
Reflective Practitioner, Jossey-Bass: San
Francisco.
FOOTNOTES
1 Our use of the term was inspired by [2], although our
training wheels are both prescriptive and restrictive.
Return to text
2 For those interested in technical specifications, the current
FLUID system is implemented in HyperCard 2.2, with video
clips (about 150) incorporated as QuickTime(tm) movies
through VideoFusion(tm).
Return to text
3 We use the term Issues where DSA uses Questions.
Return to text
4 Technically, it should be a User Test group or a Prototyper
returning this information, but for simplicity we chose to
have the learner interact directly with the Users.
Return to text
5 The Scenario screen is not shown in this paper.
Return to text
