Fundamentals of Human-Computer Interaction

2020-2021

Lectures: building 640 (PUIO), room E105 (Thursday 2pm-3:30pm) map
Instructor: Michel Beaudouin-Lafon mbl@lri.fr
TA sessions: building 640 (PUIO), room E105 (Thursday 4pm-5:30pm)
Teaching Assistant: Raphaël James web site

Blackboard Collaborate link for remote attendance: https://eu.bbcollab.com/guest/cb206fb4cc674a4bb5c3015fa8c32726

Week 1: Lecture Thursday 10 September, 2pm-3:30pm, E105 & online / TA 4pm-5:30pm, E105 & online
Week 2: Lecture Thursday 17 September, 2pm-3:30pm, E105 & online / TA 4pm-5:30pm, E105 & online
Week 3: Lecture Thursday 24 September, 2pm-3:30pm, E105 & online / No TA this week
Week 4: TA Thursday 1 October, 2pm-3:30pm, E105 & online / Lecture 4pm-5:30pm, E105 & online
Week 5: Lecture Thursday 8 October, 2pm-3:30pm, online / TA 4pm-5:30pm, online
Week 6: Lecture Thursday 15 October, 2pm-3:30pm, online / TA 4pm-5:30pm, online
Week 7: Lecture Thursday 22 October, 2pm-3:30pm, E105 & online / TA 4pm-5:30pm, E105 & online

Week 8: Lecture Thursday 5 November, 9am-10:30pm, online / TA 11am-12:30pm, online
Week 9: Lecture Thursday 12 November, 9am-10:30pm, online / TA 11am-12:30pm, online
Week 10: Lecture Thursday 19 November, 9am-10:30pm, online / TA 11am-12:30pm, online
Week 11: Lecture Thursday 26 November, 9am-10:30pm, online / TA 11am-12:30pm, online
Week 12: Lecture Thursday 3 December, 9am-10:30pm, online / TA 11am-12:30pm, online
Week 13: Lecture Thursday 10 December, 9am-10:30pm, online / TA 11am-12:30pm, online
Week 14: Lecture Thursday 17 December, 9am-10:30pm, online / TA 11am-12:30pm, online

Course Summary

This is a mandatory course between for the first year of the Interaction Specialty of the Master in Computer Science and the EIT Digital Master in Human-Computer Interaction and Design. It is also open to students from other specialties of the Master in Computer Science of Université Paris-Saclay and to the students of the Interaction, Graphics and Design Masters of Institut Polytechnique de Paris.

The course gives an overview of the field of human-computer interaction, including a short history of HCI, a review of interaction styles, basic elements of psychology and software engineering for HCI, an in-depth analysis of graphical interaction, a sampling of post-WIMP interaction techniques, the notion of conceptual modeling, and a range of theories and models for HCI.

This page and its content are solely for students enrolled in the 2020-2021 HCID and Interaction Masters at Université Paris-Saclay. It contains instructor's material (slides and audio/video recordings) that are not to be distributed without the author's written consent, and copyrighted materials (such as articles) that are only made available under the fair use exception to copyright law.

Week 1 — 10 September 2020 — Introduction, Interaction styles

Handouts

• Introduction
• Interaction styles (up to slide 8)

Recordings

• Lecture
• TA

Videos shown in class

• Myron Krueger - Video Place (1988)
• Rich Bolt / Chris Schmandt - Put-that-there (1979)
• Pierre Wellner - Digital Desk (1993)
• Otmar Hilliges (Microsof) - Holodesk (2012)
• Microsoft - RoomAlive (2014)
• Chris Harrison (CMU) - Skinput (2016)
• MIT Media Lab - InForm (2013)
• Mathieu Le Goc (Inria) - Zooids (2017)

Readings for Introduction

• Interaction is more Powerful than Algorithms. Peter Wegner, Communications of the ACM, 1997
• Interacting in Chaos. Dan Olsen, ACM Interactions, 1999
• HCI, Natural Science and Design: A Framework for Triangulation Across Disciplines. Mackay & Fayard, Designing Interactive Systems, 1997

Readings in Interaction Styles

• Direct Manipulation Interfaces. Ed Hutchins, James Hollan & Don Norman, HCI Journal, 1985.
• Chunking and Phrasing and the Design of Human-Computer Dialogues. Bill Buxton, Proc. Proc. IFIP World Congress, 1986.
• Direct Manipulation vs. Interface Agents. Ben Shneiderman & Pattie Maes, ACM Interactions, 1997.
• Augmented Reality: Linking Real and Virtual Worlds. Wendy Mackay, Proc. Advanced Visual Interfaces, 1998.
• Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. Hiroshi Ishii & Brygg Ullmer, Proc. ACM Human Factors in Computing Systems (CHI'97), 1997.

Videos about Gesture-based Interaction

• Bill Buxton - Multitouch tablet (1985)
• Minority report (2002) - scene with gesture-based interface
• Jeff Han - TED talk on multitouch interface (2006)
• Leap Motion (2012)

Week 2 — 17 September 2020 — Interaction styles, History

Handouts

• Interaction styles (from slide 9)
• History

Recordings

• Lecture
• End of lecture after the network went down
• TA

History videos (briefly) shown in class

• SketchPad
• NLS/Augment short segment on groupware - Full video of The "mother of all demos" - Edited version
• Xerox Star introduction video - Video of the Star User Interface (1984) - Video of the "Last Xerox Star Demo" (1998)

Note: some videos don't play in the browser and must be downloaded to view them with VLC.

Readings about History of HCI

Papers about history of HCI:

• The Xerox Star: a Retrospective. Jeff Johnson et al., IEEE Computer, 1989
• The Making of the Mouse. Alex Soojung-Kim Pang, Invention&Technology, 2002.
• Inventing the Lisa User Interface. Roderick Perkins, Dan Smith Keller and Frank Ludolph, ACM Interactions, 1997 (see the figures here)
• A Brief History of Human-Computer Interaction Technology. Brad Myers, ACM Interactions, 1998
• (In French) 50 ans d'interaction homme-machine : retours vers le futur. Michel Beaudouin-Lafon, Interstices, 2016.

Papers about visions:

• As we May Think. Vannevar Bush, The Atlantic Monthly, 1947
• Augmenting Human Intellect: A Conceptual Framework. Douglas Engelbart, SRI Report, 1962 (original report)
• The Computer for the 21st Century. Mark Weiser, Scientific American, 1991
• Interacting in Chaos. Dan Olsen, ACM Interactions, 1999

About innovation and invention:

• Innovation vs. Invention. Bill Buxton, Business Week, 2005
• The Long Nose of Innovation. Bill Buxton, Business Week, 2009
• Modeling and Forecasting the Information Sciences. Brian Gaines, Information Sciences, 1999

"Vision" videos to watch

• Apple Knowledge Navigator (1987)
• Microsoft Future Vision 2009
• Microsoft Future Vision 2011

Week 3 — 24 September 2020 — Psychology 101

Handouts

• Psychology 101 (up to slide 22)

Recording

• Lecture

Videos shown in class

• Monkey business illusion
• Toolglass and Magic Lenses: The See-Through Interface - Paper, E. Bier et al., Proc. ACM SIGGRAPH 93, 1993.

Readings in Psychology

General psychology:

• Behaviorism
• Cognitivism
• Gestalt perception
• James J. Gibson

Visual system:

• Held & Hein's experiment
• Preattentive processing (from Perception in Visualization)
• Change blindness (from Perception in Visualization) - See also this video and this one

Visual illusions:

• Ames room - explanation
• Mighty Optical Illusions
• Hermann's Grid (my personal favorite) - see explanation here

Auditory illusion:

• Shepard-Risset tone: an infinite glissando (see also the wikipedia page)

Week 4 — 1 October 2020 — Psychology 101 - Graphical interaction

Handouts

• Psychology 101 (from slide 22)
• Graphical Interaction (up to slide 5)

Recordings

• TA
• Lecture

Videos shown in class

• Indirect interaction

Additional resources

Motor system:

• The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement. Paul Fitts. Journal of Experimental Psychology, 1954. (See also the wikipedia page)
• Fitts' Law as a Research and Design Tool in Human-Computer Interaction. Scott Mackenzie. HCI Journal, 1992.
• Asymmetric Division of Labor in Human Skilled Bimanuel Action: The Kinematic Chain as a Model. Yves Guiard, Journal of Motor Behaviour, 1987.

Cognitive biases (general):

• Wikipedia page
• Wikipedia list of cognitive biases
• Wikipedia page on Behavioral Economics
• Cognitive bias cheat sheet. Buster Benson, Medium, sep 2016.
• 18 Cognitive Bias Examples Show Why Mental Mistakes Get Made. Jeff Desjardins, Visual Capitalist, 2018.
• The Cognitive Biases Tricking Your Brain. Ben Yagodo, The Atlantic, 2018.
• Thinking Fast and Slow. Daniel Kahneman, 2011.

Cognitive biases in HCI:

• Decision Frames: How Cognitive Biases Affect UX Practitionerse. Kathryn Whitenton, Nielsen Norman Group, 2016.
• Priming and Anchoring Effects in Visualization. Calero Valdez et al. IEEE TVCG, 2017.
• Mitigating the Attraction Effect with Visualizations. Dimara et al., IEEE TVCG, 2019.

Week 5 — 8 October 2020 — Graphical interaction

Handouts

• Graphical Interaction (from slide 6 to slide 24)

Recordings

• Lecture
• TA

Videos shown in class

• Marking menus - article - Gordon Kurtenbach's thesis
• Octopocus - article - web site

Additional resources

• A Personal History of Modeless Text Editing and Cut/Copy/Paste. Larry Tesler, Interactions Magazine, 2012.
• All the widgets - 1990 video by Brad Myers

Week 6 — 15 October 2020 — Instrumental Interaction

Handouts

• Graphical Interaction (from slide 25)
• Instrumental Interaction (up to slide 33)

Recordings

• Lecture
• TA

Videos shown in class

• 3 palettes

Videos to watch

• CPN200

Readings on instrumental interaction

• Instrumental Interaction: An Interaction Model for Designing Post-WIMP Interfaces, Beaudouin-Lafon, ACM CHI 2000.
• Reification, Polymorphism and Reuse: Three Design Principles for Designing Visual Interfaces, Beaudouin-Lafon & Mackay
• The Architecture and Implementation of CPN2000, A Post-WIMP Graphical Application, Beaudouin-Lafon & Lassen, ACM UIST 2000. CPNtools web site.

Week 7 — 22 October 2020 — Instrumental Interaction

Handouts

• Instrumental Interaction (from slide 34)

Recordings

• Lecture
• TA

Videos shown in class

• Side Views
• StickyLines
• Para
• Object-Oriented Drawing
• Textlets

Readings on instrumental interaction

• Beyond Snapping: Persistent, Tweakable Alignment and Distribution with StickyLines, Ciolfi Felice, Maudet, Mackay & Beaudouin-Lafon, ACM UIST 2016. Presentation
• Side views: persistent, on-demand previews for open-ended tasks, Terry & Mynatt, ACM UIST 2002.
• Supporting Expressive Procedural Art Creation through Direct Manipulation, Jacobs et al., ACM CHI 2017. web site.
• Object-Oriented Drawing, Xia, Araujo, Grossman, Wigdor, ACM CHI 2016. Presentation
• Textlets: Supporting Constraints and Consistency in Text Documents, Han, Renom, Mackay, Beaudouin-Lafon, ACM CHI 2020.

Demos

• Indirect interaction
• Instrumental Search
• Magnetic Guidelines
• Zip archive containing 3 Tcl/Tk demos that can be run from the terminal, assuming you have installed the Tcl/Tk (installed by default on Mac OS X):
  % wish indirect.tcl   - direct vs. indirect manipulation
  % wish search.tcl     - instrumental text search
  % wish guidelines.tcl - magnetic guidelines

Week 8 — 5 November 2020 — Pointing

Handouts

• Pointing techniques

Recordings

• Lecture
• TA

Videos on pointing and navigation shown in class

• Drag'n'pop
• Semantic pointing - live demo (click the demo link to get to the Java applet)
• Bubble cursor
• Dynaspot
• Pad++
• OrthoZoom
• Crossy

Readings on pointing and navigation

• Fitt's Law as a Research and Design Tool in Human-Computer Interaction, I. Scott MacKenzie, Human-Computer Interaction, 1992.
• "Beating" Fitts' law: Virtual Enhancements for Pointing Facilitation, R. Balakrishnan, Int. J. Human-Computer Studies, 2004.
• Semantic Pointing: improving target acquisition with control-display ratio adaptation, Blanch, Guiard & Beaudouin-Lafon, CHI 2004.
• Bubble cursor - enhancing target acquisition by dynamic resizing of the cursor's activation area, Grossman & Balakrishnan, CHI 2005 - web page - Chrome extension.
• Dynaspot - Speed-dependent Area Cursor, Chapuis, Labrune & Pietriga, CHI 2009.
• Pad - An Alternative Approach to the Computer Interface, Perlin & Fox, SIGGRAPH 1992.
• Orthozoom - 1D Multi-Scale Navigation, Appert & Fekete, CHI 2006 - web page.
• Crossy - A crossing-based drawing application, Apitz & Guimbretiere, UIST 2004 - web page.

Week 9 & Week 10 — 12 & 19 November 2020 — Conceptual modeling

Handouts

• Conceptual modeling

Recordings

• Lecture (12 nov)
• TA (12 nov)
• Lecture (19 nov)
• TA (19 nov)

Readings

• Don Norman - The Design of Everyday Things
• Carelman - Catalogue des objets introuvables - see also images from the original edition or a Google Image search (click "Images" if you get the regular Google result page).

Week 11 & Week 12 — 26 November & 3 December 2020 — Theories and Models

Handouts

• Theories and Models

Recordings

• Lecture (26 nov)
• TA (26 nov)
• Lecture (3 dec)
• TA (3 dec)

Readings

• Flat Earth
• Heliocentrism and the Copernician revolution. Copernicus, Kepler and Galileo
• Karl Popper. His most famous book:
  The Logic of Scientific Discovery
• HCI Models, Theories, and Frameworks, John Carroll (Ed). Available also on ScienceDirect. I used this book extensively to prepare this lecture.
• Science in action, Bruno Latour - an empirical study of science and technology.

Action and perception:

• Pre-attentive processing - infovis wiki.
• Gestalt perception
• Ecological Psychology. James J. Gibson's seminal book:
  The Ecological Approach to Visual Perception (also on Google Books)
• Affordances vs. Signifiers by Don Norman
• Hick's law (also known as Hick-Hyman law). Research papers:
  On the rate of gain of information, W.E. Hick, Journal of Experimental Psychology, 1952.
  How Relevant is Hick's Law for HCI?, W. Liu, J. Gori, O. Rioul, Y. Guiard, M. Beaudouin-Lafon, Proc. ACM CHI'20, 2020.
• Fitts' law. Research paper:
  The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement, P. Fitts, Journal of Experimental Psychology, 1954.
  Fitts' Law as a Research and Design Tool in Human-Computer Interaction. Scott Mackenzie, HCI Journal, 1992.
• Kinematic chain - this entry describes the general concept of a kinematic chain.
  Asymmetric Division of Labo in Human Skilled Bimanuel Action: The Kinematic Chain as a Model, Y. Guiard, Journal of Motor Behaviour, 1987. Research paper about kinematic chain and bimanual interaction.
  A study in two-handed input, W. Buxton and B. Myers, Proc. ACM CHI'86, 1986. The first study of bimanual input (HTML version).
• Toolglass and Magic Lenses: The See-Through Interface, E. Bier et al., Proc. ACM SIGGRAPH 93, 1993. An advanced bimanual interface using Guiard's principles. Video

Cognition and behaviour:

• Action Theory, from The Design of Everyday Things, Don Norman, 2002. revised edition.
• Plans and Situated Action, Lucy Suchman. PDF version.
• Human-Machine Reconfigurations, recently revised version of the above.
• Activity Theory
• Beyond the Interface: Encountering Artifacts in Use, L. Bannon and S. Bodker, in Designing Interaction: Psychology at the Human-Computer Interface, 1991. Introduced Activity Theory into HCI.
• Acting with Technology: Activity Theory and Interaction Design, V. Kaptelenin and B. Nardi, 2009. A recent book on activity theory and HCI.
• The Human-Artifact Model - an Activity Theory Approach to Artifact Ecologies, S. Bødker and C. Klokmose, Human-Computer Interaction, 2011.
• Cognitive Dimensions of Notation. Research paper:
• Notational Systems – the Cognitive Dimensions of Notations framework, A. Blackwell and T. Green, in HCI Models, Theories and Frameworks, 2003. Web site with additional resources.

Interaction:

• A Morphological Analysis of the Design Space of Input Devices, S. Card et al., ACM Trans. Office Information Systems, 1991.
• GOMS, Keystroke-level model
• The Psychology of Human Computer Interaction, S. Card, T. Moran and A. Newell, 1983.
• Project Ernestine: A validation of GOMS for prediction and explanation of real-world task performance W. Gray et al., Human-Computer Interaction, 1993.
• Extensions of GOMS analyses to expert performance requiring perception of dynamic visual and auditory information, B. John, Proc. ACM CHI'90, 1990.
• The UAN: a user-oriented representation for direct manipulation interface designs, R. Hartson, A. Siochi and D. Hix, ACM Trans. Office Information Systems, 1990.
• SwingStates: Adding state machines to Java and the Swing toolkit, C. Appert and M. Beaudouin-Lafon. Jal Software Practice and Experience, 2008. Web site with downloadable library and examples.

Software engineering:

• User Interface Management Systems: Present and Future, M. Beaudouin-Lafon, Eurographics, 1994.
• MVC, see also history of MVC
  A cookbook for using the model-view controller user interface paradigm in Smalltalk-80, G. Krasner and S. Pope, Jal Object-Oriented Programming, 1988.
• PAC
  PAC, an Object-Oriented Model for Dialog Design, J. Coutaz, Proc. INTERACT 87, 1987.

• Introduction

Recordings

• Lecture
• TA

Videos shown in class / to watch

• 2001 A Space Odyssey - picturephone sequence (1968)
• Colab - presentation of the Xerox PARC Colab (1987)
• SASSE - A collaborative writing tool (1994) - See also the video transcript

Note: some videos may not play in the browser and must be downloaded to view them with VLC.

Readings

Books:

• Readings in Groupware and Computer-Supported Cooperative Work Baecker (ed), Morgan Kaufmann, 1993.
• Computer-Supported Cooperative Work, Beaudouin-Lafon (ed), Wiley, 1999

Articles about the history and defintion of the field:

• CSCW: Four Characters in Search of a Context. Liam Bannon and Kjeld Schmidt, Proc. European CSCW Conference, 1989
• Groupware - Some Issues and Experiences. C.A. Ellis, S.J. Gibbs and G.L Rein, Communications of the ACM, 1991
• Computer-Supported Cooperative Work: History and Focus. Jonathan Grudin, IEEE Computer, 1994
• Groupware and Social Dynamics: Eight Challenges for Developers. Jonathan Grudin, Communications of the ACM, 1994

Week 14 — 17 December 2020 — Mediated Communication

Handouts

• Mediated Communication

Recordings

• Lecture
• TA

Videos shown in class / to watch

• Hydras (Sellen et al., 1992)
• VideoDraw (Tang & Minneman, 1990)
• VideoWhiteboard (Tang & Minneman, 1991)
• Clearboard (Ishii et al., 1992)
• HyperMirror - Short clip (Morikawa et al., 1998)
• Hole-in-space (Galloway & Rabinowitz, 1980)
• Interliving project overview
• VideoProbe
• MirrorSpace concept videos: transparency, blurring
• MirrorSpace exhibits: Exhibit 1. Exhibit 2. Exhibit 3

Readings

Books :

• The Information, James Gleick (2012) - this book covers much more than mediated communication, but has fascinating stories about the development of pre-telephone communication systems
• Video-Mediated Communication, Finn, Sellen & Wilbur (1997)
• Media Spaces – 20+ Years of Mediated Life, Harrison, 2009
• Awareness Systems, Markopoulos, De Ruyter and Mackay (eds), Springer, 2009.

Articles:

• Media spaces: bringing people together in a video, audio, and computing environment, Bly, Harrison & Irwin, Communications of the ACM, 1993
• Media Spaces: Environments for Informal Multimedia Interaction, Mackay, in Computer-Supported Cooperative Work, Beaudouin-Lafon (ed), Wiley (1999)

Interliving project:

• Technology probes: inspiring design for and with families, Hutchinson, Mackay, Westerlund, Bederson, Druin, Plaisant, Beaudouin-Lafon, Conversy, Evans, Hansen, Roussel, Eiderbäck, in Proc. ACM CHI, 2003.
• VideoProbe: Sharing Pictures of Everyday Life, Conversy, Mackay, Beaudouin-Lafon and Roussel, Technical Report, 2005. French version: Partager les images de la vie quotidienne et familiale avec videoProbe, Conversy et al., in Proc. IHM 2003.
• Proximity as an Interface for Video Communication, Roussel, Evans and Hansen, IEEE Multimedia, 11(3):12-16, July-September 2004.