Wellner’s Digital Desk Paper Prototype

Despite the PC revolution in the 1970s and 1980s, the use of paper in offices was only increasing. Unlike digital documents, paper is universal, cheap, portable, annotatable, easy to navigate, stack, and hold in both hands, and our musculoskeletal system is very well accustomed to it. I am personally a huge fan of paper and have leveraged it in many projects as a cheap and reliable interface (check out 1, 2, 3, and 4 for example).  This project investigates yet another use for design paper in interaction design: as a low fidelity prototype technique to recreate a seminal HCI work.

Pierre Wellner developed the Digital Desktop project in 1991. It was novel for it rejected replacing paper with computers. Instead of forcing the desktop metaphor on the computer to make it more like paper, he called for enhancing paper with the computer, giving paper digital properties and access to the computer, and introducing computer-based interactions to the physical desk such that no desktop computer is needed anymore.

Wellner’s Digital Desk encompassed a camera and projector mounted over a horizontal surface/desk. The camera tracked finger movements across the desk to detect gestures and selections, and a small microphone under the desk was proposed to better detect taps/clicks. The projector superimposed digital annotations, images, and numbers on the paper documents. A higher resolution camera focused on a small area on the desk to “read’ tasks involving intricate details such as small fonts on receipts. The large desk and top-mounted gear allowed physical objects such as paper, books, stationary, coffee cups, etc. to be naturally spread out and for computer assisted interactions to encompass the entire desk. Wellner’s video demonstrates a calculator, paper spreadsheet, text translation, keyboard typing, writing, and free-hand sketching. He also introduces a vision for concurrent editing where actions made by a remote peer are projected on the local desk.

LOW FIDELITY PROTOTYPE

The prototype demo below showcases the novelty of eradicating the computer and its desktop metaphor through a desk so ordinary, messy, tangible, and technology-free (save for the mounted camera and projector). I used a spotlight to convey the strong light Wellner’s projector would have emitted. I placed the keyboard underneath a stack of publications to emphasize its secondary role. 

The video below runs through the scenario of a student putting together an event report and budget to showcase the following Digital Desk functionalities:

1. Copying a figure from a paper magazine to the report
2. Adding figure caption by typing on the keyboard
3. Copying expenses and dates from physical receipt to the budget table
4. Selecting expenses and finding the total 
5. Printing the final result

There were no special effects or tricks used – the demo was engineered to give the illusion of an actual digital desk:

Prototype Components

The prototype consists of paper printed on cardstock and hollowed out where the figure caption and budget sum would appear, receipt with magnets glued at the back and paper snippets (for the price and date) attached to it with thin metal rods, marker with mounted magnet, green transparency paper for selecting values, and SUM and PRINT paper tags.

Power Hierarchies

To create parts that can move and then adhere somewhere else, I used a combination of polarized strong magnets (figure 5), less-strong magnets, less strong magnets with double sided tape wedged between them and their object, and thin metal rods. For example, when copying from receipts, the strong magnet on the marker attracted the price snippet (which had a thin metal rod attached to its back) because the magnet glued to the back of the receipt was weaker. When the user carries the price snippet to the budget table, the magnet mounted on the back of the table faces the same pole as that on the marker, so the marker is repelled but the price snipped can attach to either one (a little wedge from the human finger helps it slip to the budget table).

The seemingly simple task of copying and moving objects was not that easy to implement and required many considerations. The same applies for its digital equivalence: Wellner discusses many complications with camera tracking, the need for digital copies of physical documents for the concealed computer to read their content accurately, and the challenge of dealing with shadows. So that could be an answer for “why do we not have this already?” It is disappointing though that recent works inspired by the Digital Desks tend to focus on the digital (such as Curve), and hybrid paper-digital interfaces are decreasing in popularity and are mostly found in HCI4D work these days. 

Z Axis/Index: 

A challenge I encountered was trying to conceal the “bulginess” of the prototype components given how flat and intangible the projected digital documents they were supposed to represent are. Other than the fact that it was glued to the desk, my “digital” document did not only look no different than the other paper documents, but it also had several layers of paper, cardstock ribbons, carvings, double sided tape, magnets, and metal pieces underneath (figures 6 and 7). That, along with the strong light projected resulted in pronounced shadows (figure 8) that emphasized the physicalness and tangibility of the prototype. That left me frustrated because it contrasted drastically with the “intangibility” I was seeking to emulate and differentiate from normal paper documents. But given how well the audience reacted to the demo, may be shadows are not as vile  as I thought. Furthermore, do we not deliberately add shadows to UI components these days to give them depth? Perhaps the stark flatness of Wellener’s digital documents was an extreme. Flatness certainly helped distinguish the digital from the real in his work, but is that desirable given the fluid experience he was after? And maybe he could have used “more life” to their “flat UI design”.