Getting physical

Introduction

With the ingress in the market of products like Nintendo Wii, Apple iPhone and Microsoft Kinect, developers finally started realizing that there are several ways a person can control a computer besides keyboards, mice and touch screens. These days there are many alternatives, obviously based on hardware sensors, and the main difference is the dependency on software computation. In facts solutions based on computer-vision (like Microsoft Kinect) rely on state of the art software to analyze pictures captured by one or more cameras.
If you are interested on the technical side of it, I recommend to have a look to the following projects: Arduino, Processing and OpenFrameworks.

Usage with Ubuntu

During a small exploration we did internally few months ago, we thought about how Ubuntu could behave if it was more aware of its physical context. Not only detecting the tilt of the device (like iPhone apps) but also analysing the user’s presence.
This wasn’t really a new concept for me, in 2006 I experimented with a user proximity sensitive billboard idea. I reckon there is a value on adapting the content of the screen based on the distance with who is watching it.

We came up with few scenarios which are far to be developed and specified, hopefully will just open some discussions or, even better, help to start some initiatives.

Lean back fullscreen

If the user moves further from the screen while a video is playing on the focused window, the video will go automatically to fullscreen.

Fullscreen notifications

If the user is not in front of the screen, the notifications could be shown at fullscreen so the user can still read them from a different location.

Windows parallax

Since this is the year of 3D screens, we couldn’t omit a parallax effect with the windows. A gesture of the user could also trigger the launcher appearance (see prototype below).

Prototype

With few hours available, I mock up something very quickly in Processing using a face recognition library (computer-vision).
Despite it could be hard to detect the horizontal position of the user’s head without a camera, we are in no way defining the technology required. The proximity could be in-facts detected with infra-red or ultra-sound sensors.

Parallax and fullscreen interaction via webcam from Canonical Design on Vimeo.