The world is your computer

Computing is starting to think outside the box as researchers from the Massachusetts Institute of Technology (MIT) launch a “wearable gestural interface” that can turn your wrist into a watch, your hands into a photo frame or your newspaper into a TV. SixthSense comprises a pocket projector, mirror and camera – as well as some very clever software – developed by the Fluid Interfaces Group at MIT’s Media Lab.

The system was unveiled at the Computer-Human Interaction (CHI) conference in Boston, April 4 – 9. Researchers at MIT’s Fluid Interfaces Group, led by Dr Pattie Maes, said the idea behind SixthSense was to create a link between our digital devices and our interactions with the physical world. The team wanted to free information from its traditional confines of paper or a digital screen. The prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects to be used as interfaces; while the camera recognises and tracks user's hand gestures and physical objects using computer-vision based techniques.

The software program processes the video stream data captured by the camera and tracks the locations of the coloured markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction. The SixthSense prototype implements several applications.

The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. The system is also designed to recognise a user’s freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture.

The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognises those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check their mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analogue watch.

The current prototype system costs approximate $350 to build.

A new era in trip planning

Ok, so far when planning a trip you do your research on the internet and dig for sites with tourist info and reviews from people who has been there. The next step is just selecting a location and then create your trip based on the pictures and travel paths of other. Since most digital cameras will be GPS enables in the near future (there are some already) and people use the ability of using tags more and more you just creat a trip based upon tags. So for example going to a city then look at the map and filter a trip for party or shopping or dining and you see where most people go including pictures of that area (and reviews)

Imaging doing this on your multitouch dining table:)

Anyway, planning your trip has come closer after some cleaver stuff at MIT

(un)photographed Spain
When posting photos online, users of the photo sharing platform Flickr transmit to the world their perspective of a place or event through the lens of a digital camera. Each digital photo file codes both the time when that photo was taken and the location it captures. Analyzing this information allows us to follow the trail that each Flickr photographer travels through Spain. (Un)photographed Spain maps thousands of these public, digital footprints over one year. As photos overlap in certain locations, they expose the places that attract the photographer's gaze . In contrast, the absence of images in other locations reveal the unphotographed spaces of a more introverted Spain.

Density and flows of photographers
In 2007, the Iberian Penninsula clearly splits between Portugal and Spain with Barcelona constantly acting as main hub connecting people to Madrid, the Balearic Islands, Andalucia and the Cantabrian Sea in the North. hi-res images > low-res video


(un)photographed Spain from senseablecity on Vimeo.

spaces of diversity
About 60% of Flickr users disclose information on their home country. Analysis of the time and location data embedded in their digital photo files allows us to examine the Flickr photographers' geographic presence and trails over time, and to differentiate locals from visitors. Based on this information, spaces of diversity maps the contrast between where locals capture images and where visitor communities such as Britons experience of Spain. The yellow lines reveal the most common paths photographers follow as they capture images in different places.

Britons weaving their path in Barcelona
Britons who visited Barcelona in Fall 2007 stayed on the beaten paths delimited by the city's main elements such as Parc Guel and Sagrada Familiar with Passeig de Gracia and Rambla acting as artery. The photos also confirms their pleasure for football (Camp Nou) parties (Forum) and the mediteranean sea (Barceloneta). > hi-res images > video


Spaces of diversity (where are the britons) from senseable on Vimeo.

spaces of activity
Photographers often attach descriptions and tags when posting their photos on Flickr. The data mining of these tags allows us to infer the kinds of activities that these photos capture. Spaces of activity reveals the regions and cities that host memorable parties in Spain over the course of a year. It also shows the places that attract more quiet experiences through art. The size of the glow on the map corresponds to the density of each type of activity. A greater glow implies a greater density of photographic activity.

Partying in Barcelona
This animation of the photos with tags related to "partying" in Summer 2007 shows that Barcelona confines its fun to the old town (Ciutat Vella) known for its high density of tourists, the bohemian distric of Gracia and the Forum area and its music festivals.> hi-res images > video


Spaces of activity (where are the parties?) from senseable on Vimeo.