Kids in Paris making music on an iPad by simply moving their hands through sand. Apple
APPLE'S "1.24.14" VIDEO #30YEARSOFMAC #MAC30
[subtitle]Hey User Studio, so what's up, you're designing controllers now?![/subtitle]
We've always been designing controllers!
But true, it's usually pure, graphical controllers that you can click or tap on your screen. This time we've gone tangible. The constraints are a bit different when you design interfaces out of wood, metal and electronics rather than pixels and code, but ultimately it's the same job: interaction design.
[subtitle]All right, we believe you ;) Then why this controller?[/subtitle]
There are tons of kinds of controllers nowadays that enable us, humans, to interact with machines: those are of course the keyboard, the mouse, the joystick, the gamepad, the touch screen... but also the Wiimote or your car's steering wheel.
[fullwidth]
[/fullwidth]
[quotebox][quote]In the real world you can't move a grain of sand back to its previous position (or can you?!), and you definitely get you hands dirty...[/quote][author][/author][/quotebox]
All of these tangible user interfaces have in common that you can, more or less, go back to a previous, default position... They also share their cold, plastic cleanliness: you don’t need to wipe your fingers after using them! These interfaces don't change, they don't evolve while being used, they're stable and all of that is on purpose. Their behavior is infinitely replicable and accurately predictable.
Both of these properties – the default position and the cleanliness – tend to separate the interactions with these devices from “real world” interactions: in the real world you can't move a grain of sand back to its previous position (or can you?!), and you definitely get you hands dirty...
[fullwidth]
[/fullwidth]
[trivia]A 570cm3 dish that contains about 8.600 seeds of dry tapioca grains[/trivia]
Our research led us to wonder if and how we could change the relationship that humans have with tangible controllers: at the time (2011) we were working on trying to control thousands of particles on the screen in the most natural, intuitive fashion possible. We figured there was no better way than by actually controlling real world particles! So when creating this new "DIRTInterface", we set our minds on making something a little less accurate, while a lot more subtle, constantly adapting, almost alive. Tackling the cold, abrupt interaction that traditional controllers impose on us... It was all about interaction design politics ;)
[subtitle]Ok, so what's DIRTI in the first place?[/subtitle]
[quotebox][quote]It's the World's first tapioca interface![/quote][author][/author][/quotebox]
It's the World's first tapioca interface! No really, it enables you to control your computer or your iPad with tapioca or anything else that's semi-transparent and that you can mold, like vanilla ice cream for example. Any non-opaque material that's either granular or liquid will do just fine. It's kind of a real-world interface. And the acronym stands for Dirty Tangible Interface. Tacky? Yeaaaah, we love tacky!
[fullwidth]
[/fullwidth]
[trivia]2.3 billions of little sand rocks, or 37 billions of billions of water molecules[/trivia]
You, the user, interact with your machine by moving the material around in a sand-blasted dish. Anything that you're going to produce from within the Dirty Tangible Interface can not be 100% accurate, but it's infinitely refined, expressive and subtle. And you can't cancel any action or go back to a previous, default position, but you can control any graphics or sounds coming out of your machine with amazing expressivity, just like with real world instruments. Say, a violin. Not even kidding. And who wouldn't like plunging their hands in ice cream?!
[ratioresize][/ratioresize]
[subtitle]Got it. Then what's DIRTI for iPad's specificity?[/subtitle]
DIRTI for iPad is the compact, integrated product version of the Dirty Tangible Interface. It's super easy to setup and super easy to use. Especially for kids for whom using touch tablets is practically a second nature!
In the particular case featured here, during the workshop at the Maison des Petits of the CENTQUATRE in Paris, the focus was on linking the movements within DIRTI to graphics and sounds produced by a specially-brewed iPad app:
[ratioresize][/ratioresize]
The iPad's Graphical User Interface is right there under the user's fingers, and the relationship between the graphics of the application that we developed and what happens within the glass dish is self-explanatory. Same goes for the sounds. The sound part of the app was created and developed by researcher Diemo Schwarz from IRCAM with whom the DIRTI concept has evolved from the start, and sounds were composed by sound designer Roland Cahen.
[subtitle]Sweet... so how does it work?[/subtitle]
The way it works is as follows: within the box it's pitch black. We've placed a webcam right underneath the sand-blasted glass dish that contains the interaction material, be it tapioca or vanilla ice cream. That camera is plugged into a tiny computer that's called a Raspberry Pi. Together they track the movements and the density of the interaction material and transmit this information to the iPad… In the end, what's important is that most of the computing – the smart part – happens on the iPad.
[ratioresize][/ratioresize]
Of course the very first prototype was far from perfect: many updates were needed to reach the point we're at nowadays... it's evolved tremendously. But the philosophy behind it has not changed a bit along the process. We really see our controller as an illustration of the FabLab movement, with its identity residing in its building principles: iterative prototyping & exploration.
[subtitle]Right, so there are specific iPad apps made for the DIRTI controller?[/subtitle]
[quotebox][quote]Terrain editors for game level design, urban planning apps, science learning tools…[/quote][author][/author][/quotebox]
Definitely. And good job, obviously you're following ;) So indeed, based on the API that our team is building right now, a great palette of apps can be developed to use what comes out of our controller. Terrain editors for game level design, urban planning apps, science learning tools, the list could go on and on...
[subtitle]And for now it's focused on kids, why?[/subtitle]
When we decided to develop the iPad version of the DIRTI controller, we had one thing only on our minds: we wanted to build the simplest, least "techy", most familiar interface possible for kids to play with. At the same time, it's really a rebel controller: it encourages them to spread tapioca all over the place! We just couldn't wait until this would happen... and it did right away!
[paragraphwidth]
[/paragraphwidth]
[brandquotebox][quote]Lets young children control an audiovisual symphony by playing with a bowl of tapioca.[/quote][img][/img][author]Gizmodo[/author][/brandquotebox]
But at the same time they were really focused. Kids are the best beta-testers you can find: if it's not a blast, they're gonna let you know straight away. They just won't use it, period. But if they like it, then they really, really like it and it's the most rewarding thing! At the Maison des Petits of the CENTQUATRE in Paris, we've had kids under 4 years-old play for over an hour, on their own. It's amazing how they get it. It gives them a really close relationship with the graphics and sounds that are produced by our app on the iPad. I mean it’s wonderful to see how absorbed they are with things that they touch, things that they mould. To go back to interaction design politics, At User Studio we really believe that haptics and non-standard touch interactions are under-rated learning tools. Let's mould more stuff!
[fullwidth]
[/fullwidth]
This project was made possible thanks to the help of the genius researcher Diemo Schwarz and the humongous amount of work of the IMTR (Real-Time Music Interaction) team at Ircam, the tremendously talented sound designer Roland Cahen, the nonetheless talented composer Éric Broitmann from Motus. But also thanks to the lasers and cool professionals at WeCut, the great music of Destronics, the awesome people at the CENTQUATRE and especially Valérie Senghor & Nicole Roux, the early help of Romain Pascal, the support of Christian Jacquemin and the French National Research Agency funded group Topophonie.