Mobile Phones as Pointing Devices
Mobile Phones as Pointing Devices Rafael “Tico” Ballagas (RWTH Aachen) Michael Rohs (ETH Zurich) Jennifer Sheridan (University of Lancaster)
Overview
Introduction / Motivation
Large Public Displays
Phones vs. Other Interaction Techniques
New Phonecam Interaction Techniques
Sweep
Point & Shoot
Future Directions
Potential Improvements
Deployment Opportunities
Demo
Tico Ballagas
Mob ile Phones as Pointing Devices
2/16
Interaction with Large Displays Personal
Semi-Public Tico Ballagas
Mob ile Phones as Pointing Devices
Public 3/16
Potential Applications Large Public Displays
Games Interactive art Digital bulletin boards Advertising
Tico Ballagas
Mob ile Phones as Pointing Devices
4/16
Direct surface interaction Clear affordance One-to-one mapping High Serendipity But… Physical Security Scalability Perspective Sanitation / Maintanence Multi-user
Tico Ballagas
Mob ile Phones as Pointing Devices
5/16
Mobile Phones
Inherent Multi-user support (1 device / person) Physical Security not an issue User familiarity Connectivity standards Many built-in sensors / actuators
Tico Ballagas
Mob ile Phones as Pointing Devices
6/16
Mobile Phones for Pointing Tasks Two Camera-based Techniques Sweep Point & Shoot Nokia 6600 Phonecam
Tico Ballagas
Mob ile Phones as Pointing Devices
7/16
Sweep
Optical Flow Technique
The joystick is used as a clutch
Allows the phone to be used like an optical mouse
Allows user to reposition arm
User can focus attention on the large display
Tico Ballagas
Mob ile Phones as Pointing Devices
8/16
Tico Ballagas
Mob ile Phones as Pointing Devices
9/16
Point and Shoot
Aim using cross-hair cursor on phone screen Take a picture Item that you selected through the camera lens becomes selected on the large display.
Tico Ballagas
Mob ile Phones as Pointing Devices
10/16
Tico Ballagas
Mob ile Phones as Pointing Devices
11/16
Visual Codes
Tico Ballagas
Mob ile Phones as Pointing Devices
Arbitrary
Orientation
Tilt
Rotation
In Point & Shoot they are used to derive coordinate system on the display surface. Currently 83 bits 12/16
Example Scenario
Tico Ballagas
Mob ile Phones as Pointing Devices
13/16
Performance Evaluation
Tico Ballagas
Mob ile Phones as Pointing Devices
14/16
Improving Interactions
Strategies for combining sensors
Camera + Accelerometer
Improved Optical Flow algorithms
Higher Resolution
Faster Sample Rate
Tico Ballagas
Mob ile Phones as Pointing Devices
15/16
Deployment Opportunities
eCampus REXplorer
Tico Ballagas
Mob ile Phones as Pointing Devices
16/16
Overview
Introduction / Motivation
Large Public Displays
Phones vs. Other Interaction Techniques
New Phonecam Interaction Techniques
Sweep
Point & Shoot
Future Directions
Potential Improvements
Deployment Opportunities
Demo
Tico Ballagas
Mob ile Phones as Pointing Devices
2/16
Interaction with Large Displays Personal
Semi-Public Tico Ballagas
Mob ile Phones as Pointing Devices
Public 3/16
Potential Applications Large Public Displays
Games Interactive art Digital bulletin boards Advertising
Tico Ballagas
Mob ile Phones as Pointing Devices
4/16
Direct surface interaction Clear affordance One-to-one mapping High Serendipity But… Physical Security Scalability Perspective Sanitation / Maintanence Multi-user
Tico Ballagas
Mob ile Phones as Pointing Devices
5/16
Mobile Phones
Inherent Multi-user support (1 device / person) Physical Security not an issue User familiarity Connectivity standards Many built-in sensors / actuators
Tico Ballagas
Mob ile Phones as Pointing Devices
6/16
Mobile Phones for Pointing Tasks Two Camera-based Techniques Sweep Point & Shoot Nokia 6600 Phonecam
Tico Ballagas
Mob ile Phones as Pointing Devices
7/16
Sweep
Optical Flow Technique
The joystick is used as a clutch
Allows the phone to be used like an optical mouse
Allows user to reposition arm
User can focus attention on the large display
Tico Ballagas
Mob ile Phones as Pointing Devices
8/16
Tico Ballagas
Mob ile Phones as Pointing Devices
9/16
Point and Shoot
Aim using cross-hair cursor on phone screen Take a picture Item that you selected through the camera lens becomes selected on the large display.
Tico Ballagas
Mob ile Phones as Pointing Devices
10/16
Tico Ballagas
Mob ile Phones as Pointing Devices
11/16
Visual Codes
Tico Ballagas
Mob ile Phones as Pointing Devices
Arbitrary
Orientation
Tilt
Rotation
In Point & Shoot they are used to derive coordinate system on the display surface. Currently 83 bits 12/16
Example Scenario
Tico Ballagas
Mob ile Phones as Pointing Devices
13/16
Performance Evaluation
Tico Ballagas
Mob ile Phones as Pointing Devices
14/16
Improving Interactions
Strategies for combining sensors
Camera + Accelerometer
Improved Optical Flow algorithms
Higher Resolution
Faster Sample Rate
Tico Ballagas
Mob ile Phones as Pointing Devices
15/16
Deployment Opportunities
eCampus REXplorer
Tico Ballagas
Mob ile Phones as Pointing Devices
16/16
