Tangible Musical Interfaces for Children
Seth Garrison / First Poster Session
Tangible User Interfaces:
- Coined by Hiroshi Ishii and Brygg Ullmer in the paper “Tangible Bits.”
- TUI’s refer to devices that “augment the real physical world by coupling digital information to everyday physical objects and environments.”
- Intended to bridge the gap between the physical environment and digital technologies.
- Aims to restructure the typical GUI’s inherent in most popularized forms of Human-Computer Interaction (HCI), by making palpable controls that directly relate to the system being explored.
- Inspired by the visions set forth in “Ubiquitous Computing” and “Augmented Reality.”
Examples TUI’s in Music:
- reacTIVision: open source software coupling cameras and sensors with projection as feedback. Some examples of interfaces using this technology:
o reacTable: http://www.reactable.com/reactable/
o reacTogon: http://blog.makezine.com/archive/2008/04/reactogon_arpeggiator_tab.html
o Bubblegum Sequencer: http://www.backin.de/gumball/
- Beat Blocks: http://www.beatblocks.com/
Examples of Music-Related TUI’s for Young People:
- Mozart Magical Cube
- I-Blocks: http://www.youtube.com/watch?v=2512lIR5mwM
- Zoundz: http://www.zizzle.com/
- Quadrilla Music Melody Composer
- Toy Symphony: http://www.toysymphony.net
o Beat Bugs
o Music Shapers
I just came across Think Geek: Stuff for Smart Masses.
I’m obsessed with their toys.
Here are a few examples:
They also sell fun instruments like the Stylophone and an updated and small version of the Tenori-on. Things like this make life worth living.
Don’t worry, I’m gonna test em all out for you and make sure they’re all good.
Here’s a couple of cool non musical toys:
I leave you with this quote touting their microcontroller kit:
This Isn’t Your Father’s Electronics Kit
Your kids need valuable electronic skills to thwart the upcoming robot uprising in 2050. Are you going to let their brains languish playing spongebob on Gameboy all day… or are you going to take action and teach them some serious electronic and programming techniques?
As defined in a seminal paper by Mitchell Resnick (et al), “digital manipulatives” are “computationally enhanced versions of traditional children’s toys.” These tools are based on the concept that physical objects can “play an important role in the learning process,” and this paper provides a cogent history of their use in early childhood educational philosophies.
Johann Heinrich Pesalozzi (1746-1827): “advocates for ‘hands-on-learning,’…asserted that students need to learn through their senses and through physical activity, arguing for ‘things before words, concrete before abstract.’”
Friedrich Froebel: creates first kindergarten in 1837. Influenced by Pestalozzi, he created a set of 20 “gifts”. He carefully designed these physical objects for kids to “manipulate, recognize and appreciate the common patterns and forms found in nature.”
Maria Montesorri: founded a network of schools based on the “education of the senses.” She expanded the use of these tools with the belief that their use would enable children to “learn through personal investigation and exploration.”
Jean Piaget: “provided an epistemological foundation for these educational ideas.” He outlined a particular cognitive development of children, stressing the importance of “concrete operations” in the primary stages of knowledge formation.
This paper proceeds by highlighting the integration of new computer technologies with these traditional learning objects, suggesting that tools of this kind exist not as separate projects but as part of new family of toys.
Montessori, M. (1912). The Montessori Method. New York: Frederick Stokes Co.
In the 2005 paper presented at the CHI (Computer Human Interaction) conference, Zuckerman et al categorize tangible interfaces for eduction. They define two types: “Froebel-inspired Manipulatives” (FiMs) and “Montessori-inspired Manipulatives” (MiMs). The distinction between these two types is that while FiMs encourage children to build real-life structures, MiMs allow them to model more abstract structures or systems. While the goal of this paper is not to question the integrity of FiMs, it seeks to emphasize the increased potential of Digital MiMs to teach children fundamental aspects of complex systems. In this team’s design, these models can analogize a variety of real-world situations and encourage a higher-level learning environment through which children are invited to make their own associations.
Out of all their design guidelines, I find the “Level of Abstraction” criterion the most exciting. To quote:
(2) Level of Abstraction – maintain a high level of abstraction of the constructed simulations and structures so concreteness would come from a child’s analogies rather than a structure’s visual form (e.g. the constructed creation should not look like something familiar from real-life).
This philosophy resonates with Donald Norman’s idea that metaphors are potentially dangerous tools in technology design. In his book “The Invisible Computer,” he suggests,
It is true that the metaphor is appropriate in the initial stages of learning. But while those first stages are only there temporarily, the metaphor is with us forever.
I have to say that I agree. From a personal standpoint, every time I try to teach myself a programming language, I find that trying to associate a piece of code with a real-world occurrence does not help me to implement it. Instead, it muddies up my understanding of the conceptual model and I spend more time unlearning the metaphor than mastering the language.
It is this gap in understanding that I am interested in analyzing. The SystemBlocks and FlowBlocks designed by Zuckerman provide a way to close that gap, or to prevent it from ever existing.
Let me start by saying that Block Jam is awesome! It is an inviting, accessible and modular interface that allows multiple users to “create” and combine complex musical sequences.
HOWEVER, in light of the last post about FiMs vs MiMs, I find that Block Jam falls into the former category. The philosophy behind the device is less concerned with educating the user on the nature or structure of musical sequences than it is with providing a user-friendly way of interfacing with said structures. The developers list “What will this shape sound like?” as a desired user experience. In this light, it is less of a pedagogical tool than it is a breaking down of the traditional barriers inherent in music production. It removes the necessity of what Peter Webster defines as “enabling skills” and replaces them with an “enabling condition.”
Granted, this device does not claim to educate. In their presentation, the designers state, “‘It would be great for parties’ was a typical response. Everyone wanted to take them home.” Yet I think it’s possible for a device to entertain, to sell, to encourage and to educate simultaneously. While this interface succeeds in providing an engaging environment for musical collaboration, it does not offer as direct a route to musical expression. The screen-based authoring environment for the composition of sequences, in my opinion, represents a wrinkle in the system as a whole. I think the ability to compose the sequences in the same manner as one manipulates their location within a larger pattern would make this design stronger.
Block Jam is a musical interface controlled by the arrangement of 25 tangible blocks. By arranging the blocks musical phrases and sequences are created, allowing multiple users to play and collaborate. The system takes advantage of both graphical and tangible user interfaces. Each block has a visual display and a combination of a gestural input and a click-able input. Each Block metaphorically contains a sound group that can be chosen via the gestural input, the click-able input changes a block functionally. Thus, musically complex and engaging configurations can be rapidly assembled. The tangible nature of the blocks and the intuitive interface promotes face-to-face collaboration, and the presence of the GUI allows for remote collaboration across a network.
Henry Newton-Dunn , Hiroaki Nakano , James Gibson, Block jam: a tangible interface for interactive music, Proceedings of the 2003 conference on New interfaces for musical expression, May 22-24, 2003, Montreal, Quebec, Canada
I just came across this thesis that makes a cogent connection between constructivist theories and “tangible user interfaces.”
This abstract concisely defines the motivation behind my own investigations:
For over a century, educators and constructivist theorists have argued that children learn by actively forming and testing – constructing – theories about how the world works. Recent efforts in the design of “tangible user interfaces” (TUIs) for learning have sought to bring together interaction models like direct manipulation and pedagogical frameworks like constructivism to make new, often complex, ideas salient for young children. Tangible interfaces attempt to eliminate the distance between the computational and physical world by making behavior directly manipulable with one’s hands. In the past, systems for children to model behavior have been either intuitive-but-simple (e.g. curlybot) or complex-but-abstract, (e.g. LEGO Mindstorms). In order to develop a system that supports a user’s transition from intuitive-but-simple constructions to constructions that are complex-but-abstract, I draw upon constructivist educational theories, particularly Bruner’s theories of how learning progresses through enactive then iconic and then symbolic representations. This thesis presents an example system and set of design guidelines to create a class of tools that helps people transition from simple-but-intuitive exploration to abstract-and-flexible exploration. The Topobo system is designed to facilitate mental transitions between different representations of ideas, and between different tools. A modular approach, with an inherent grammar, helps people make such transitions. With Topobo, children use enactive knowledge, e.g. knowing how to walk, as the intellectual basis to understand a scientific domain, e.g. engineering and robot locomotion. Queens, backpacks, Remix and Robo add various abstractions to the system, and extend the tangible interface. Children use Topobo to transition from hands-on knowledge to theories that can be tested and reformulated, employing a combination of enactive, iconic and symbolic representations of ideas.
You can preview the entire thesis on MIT Media Lab’s Tangible Media Groups publication site.
A pretty complex interface utilizing reacTIVision.
From their website:
The Reactable is a revolutionary new electronic musical instrument designed to create and perform the music of today and tomorrow. It combines state of the art technologies with a simple and intuitive design, which enables musicians to experiment with sound, change its structure, control its parameters and be creative in a direct and refreshing way, unlike anything you have ever known before.
The Reactable uses a so called tangible interface, where the musician controls the system by manipulating tangible objects. The instrument is based on a translucent and luminous round table, and by putting these pucks on the Reactable surface, by turning them and connecting them to each other, performers can combine different elements like synthesizers, effects, sample loops or control elements in order to create a unique and flexible composition.
As soon as any puck is placed on the surface, it is illuminated and starts to interact with the other neighboring pucks, according to their positions and proximity. These interactions are visible on the table surface which acts as a screen, giving instant feedback about what is currently going on in the Reactable turning music into something visible and tangible.
Additionally, performers can also change the behavior of the objects by touching and interacting with the table surface, and because the Reactable technology is “multi-touch”, there is not limit to the number of fingers that can be used simultaneously. As a matter of fact, the Reactable was specially designed so that it could also be used by several performers at the same time, thus opening up a whole new universe of pedagogical, entertaining and creative possibilities with its collaborative and multi-user capabilities.
you can read about it on make:online.
it’s fascinating. if i had the chance, i would play with it for hours.
but could i afford it or access to it?
can we make a manipulative-on-a-budget? and if so, how? donald norman makes a distinction between complexity and difficulty in developing technologies. he defines complexity within the technical structure of the system and difficulty within the user’s experience. my question is how do you make the complexity and the difficulty of a technology equal? in other words, how do you make knowing how something works the same as knowing how to use it?