The field of human-computer interaction is an ever-expanding field occasioned by daily leaps and bounds in the information technology landscape. The information technology explosion of the twentieth century brought with it the perennial challenge of how to interact with computers.
With advances in computer processing speeds and with the desire to connect all people in the planet, human-computer interaction has developed to ensure that there is a way every person on earth can participate in the information technology revolution.
A standard PC provides the basic platform and continues to be the basis for most of the interaction with information technology for most people. Over the last decade, the mobile phone has become a very significant member of this club. Growth in mobile telephony has been phenomenal. This is especially true for the developing world.
Human-computer interaction presents the missing link between information technology equipment and potential users. It is no longer enough to wait for children to grow up and master using a keyboard and a mouse before integrating them to information technology. It is also insufficient to produce information technology-based toys for them.
Human-computer interaction is the bridge that links them to information technology. It should meet some practical needs in the process. Seba, Lew and Huang (2004) stated, “In many important HCI applications such as computer-aided tutoring and learning, it is highly desirable (even mandatory) that the response of the computer takes into account the emotional or cognitive state of the user” (p. 2).
The tangible bubble is one such attempt to provide a means that children whose dexterity is limited and who have not yet developed cognitive skills that they need to operate a PC with the opportunity to get involved in digital information processing in a manner consistent with their stage of development.
Tangible Video Bubble Paper Summary
Tangible video bubbles are a video based drawing space which children use to create video art that has both input and output capabilities. Its operation involves pressing a record button for voice input and squeezing the bubble at various points to playback the recording at different speeds.
The system also includes an interactive canvas. The bubble targets children aged five to seven who portray a very easy and natural interaction with different media using multiple senses. They are able to move easily from one form of media to another, rather than follow a particular path of development in their media creations.
Technological tools that capture the imagination of children and inspire their creativity include webcams, microphones, and video cameras. They show a fascination with the use of these tools and they end up recording themselves and their surroundings for sheer pleasure.
These tools however do not provide the optimal opportunity for children to construct meaning out of what they are doing with them as they do when they are using tools such as a paintbrush and markers. This gap inspired the researchers to develop a new kinesthetic multimedia tool that will encourage and facilitate children’s meaning-making as they interact with technological devices.
The tangible video bubble is flexible physical vessel that has both input and output characteristics that children use to record videos and to play them back. It also allows them to manipulate and playback the recordings in real-time, and to ‘spill out’ the recordings out onto an interactive drawing canvas in order to incorporate them into drawings initially made on the canvas.
The play space for the system includes the tangible video bubble and an interactive drawing canvas. The tangible video bubble is, “a large, soft, and huggable ball equipped with a video camera and a screen for children to record and playback video messages” (Ryokai, Raffle, Horii and Mann, 2010. p. 3).
Pressing a button at the top of the bubble and speaking into an opening on the bubble surface enables recording. In the meantime, a video camera inside the bubble records the child’s actions. Through a reflective surface inside the bubble, the child sees his/her reflection inside the bubble while doing the recording. After capturing the message, the child plays back the message by squeezing the bubble.
Different squeezing positions result in different playback possibilities. The amount of pressure used in the squeezing controls the playback speed. The greater the pressure applied, the faster the playback speed achieved and the lesser the pressure applied, the slower the playback.
To Replay specific parts of the recording, the child squeezes specific parts of the bubble. In order to transfer the recording to the interactive canvas, the child squeezes the bubble out on a spill platform. There is the option of doing this at different speeds, together with the option of spelling out a particular section of the recording. The canvas can accommodate several bubbles. The set up enables the addition of as many bubbles as desired.
The canvas has touch properties that allow the child to replay the content from the canvas. It is not possible to manipulate further the bubble once transferred to the canvas. It is possible to operate both the bubble and the canvas simultaneously, enabling more than one child at a time to use the system.
The canvas has a color pallet that allows the child to draw on it as he would on paper. This makes it possible for the child to combine the recording with a drawing creating an art form that has a message that pops out when touched.
The technical implementation of the bubble includes a USB webcam fitted with a peephole lens to maximize field of view. There are green LEDs inside the sphere with red LEDs lining the aperture. The fitting ensures that the result is uniform without regard to the direction of squeezing.
During squeezing of the sphere, the green LEDs move closer to the camera, thereby increasing the video’s green saturation. With calibration, this saturation provides a reliable squeezing sensor. There is a special preview area in the bubble, allowing for previewing any recordings made. During playback, change of speed does not alter pitch. This makes it easier for the child to understand the recorded message.
During laboratory tests on the bubble, a researcher showed children how to operate the bubble for a period of five minutes, and then left them to interact with it for as long as they wished. They did not receive any explicit practice on how to master the different playback speeds. As it turned out, some children spent more time playing with the playback speed while others preferred to draw on the interactive canvas.
The result the researchers got include the fact that the children embraced the idea of the bubble quickly, and were soon recording and playing back the records. They transitioned easily between the canvas and the bubble. They played with different playback speeds before spilling the content to the drawing canvas.
The popular application the children made of the system was to do a video phrase and then making a drawing to match it. Some played with the sound effects produced by speed alteration. Older children showed the tendency to plan their recording and drawing before implementing the project on the bubble, and they had plots that are more complex.
State of Artificial, Augmented, and Virtual Realities
There are many applications using artificial and augmented reality in the market. Novel ideas keep sprouting as companies seek to outdo competition. There is a push for more immersive technologies to enhance the digital experience. Virtual reality in digital environments provides a user with a real-life experience of an actual place from a digital platform.
Multimodal devices provide the link with these environments, making possible a sensible experience. It is possible to interact with a virtual artifact in these environments. Virtual reality is the basis for production of immersive tools such as computer-aided design software and graphics hardware acceleration.
Virtual reality forms the basis of many games, television programs, advertisements, movies, art, and marketing materials. Augmented reality refers to the digitization of one’s immediate environment to increase one’s perception of reality. Research in this field aims at interposing computer imagery to a real-life situation to enable a perceived digital manipulation of the resulting environment.
These Augmented reality environments have in them many sensors and actuators, which include cameras and projectors. This field makes possible the use of 3-D television and holograms, which improve the quality of digital experience. Studies in the field take advantage of advances in the study of human vision, neurology, and psychology.
While entertainment applications such as movie theatres, gaming, and television seem to get the first products from developments in this field, other areas such as medicine, engineering and education are catching up quickly with applications that improve problem-solving techniques.
There are a number of interesting developments involving bubble multimodal input systems in human-computer interaction. The tangible video bubble is a special application developed to take advantage of the special stage of development in children.
In the MIT Technological Review Magazine, Grifantini, reported on the most interesting projects based on this concept presented in, The Tangible, Embedded, and Embodied Interaction Conference, held in Cambridge, Massachusetts. The first among these is a physically responsive map, which comprises a tabletop, which displays 3-D shapes “on a moving, flexible surface” (Grifantini, 2010).
The display provides a 3-D feel of the surface it is representing, and can represent multiple features. In another project, the development of an interactive art display robot, called cobot because it collaborates with the artist, is in progress. Of the two Cobots presented, one draws yellow circular patterns until it receives an audio command. It then draws straight lines.
A second Cobot changes its response depending on line and shadow. In another project, Spanish researchers presented a tangible digital jukebox, which has an infrared camera and a projector. Someone is able to use a piece of paper as a playlist. A project depicting an augmented reality pattern table was also on display. In this project, users have the opportunity to interact with patterns and shapes of both digital and physical nature.
By using a projector and an infrared camera, a user can pick a physical shape. Digitization of the shape for manipulation follows. The researchers said that the project presents the possibility of developing educational tools based on augmented reality for teaching children about mathematical shapes.
The final project featured was a soap bubble display. This project demonstrated the use of a camera to track soap bubbles spitted on a soapy surface, which when moved by a finger or by blowing, controlled lights or images that projected to them.
Ying et al. (2010) developed a prototype called bubble journey, which aims at providing children with an immersive gaming experience. The user controls an avatar developed in flash by blowing a real handle. This is to promote the use of the whole body during gamming to increase the physical activity of a child playing a digital game.
Theirs was a contribution towards the drive to develop games that avoid the traditional input devices such as keyboards, joysticks, and gamepads, which do not promote activity, for novel options such as web cameras, dance pads, and exercise bikes that tend to require the user apply physical exertion to play the game.
Another demonstration of the use of bubble is the tangible message bubble. Ryokai, Raffle and Brooks (2009) developed this concept where children can use a bubble to record and send messages to loved ones and to their friends.
Their goal was to, “invent new tangible communication tools for youngsters that combine state of the art communication technologies with the physicality, simplicity, and immediacy of children’s toys” (Ryokai et al., 2009, p.2). This bubble recorded a video stream whenever someone’s face was within range and allowed for manipulation of the recorded stream, before sending it to the intended recipient.
The impacts Artificial, Augmented, and Virtual Realities in HCI
These projects demonstrate that this is an active area of research trying to take advantage of development in other fields such as learning methods, and digital possibilities arising out of information technology developments. Another common feature these projects have is the use of widely available technologies, which are becoming even more accessible because of rapidly falling costs.
They show that human-computer interaction has a lot of potential for growth, not necessarily because of new technological advancement but based on innovative application of commonplace ones.
Three fields that will benefit much from development of this area of Human-computer interaction include health, medicine, and education. Through the development of games that require a user to interact with the gaming environment with their whole body, the health of children stands to benefit.
Health complications related to a sedentary lifestyle such as obesity thrive in many western countries, which also have a very high incidence of gamming.
Currently, children spend very many hours playing video games. Since it is very difficult to stop them from playing the games to an appreciable extent, it makes sense to design games that will still provide them with the gamming pleasure, but that will simultaneously require a greater degree of physical exertion.
The field of medicine will benefit from these developments by the application of special input devices to allow persons who have artificial limbs to interact properly with technology. With an interactive device like the bubble, anyone with any condition leading to loss of dexterity of the fingers will still be able to communicate, using the squeeze as an input.
It promises new possibilities in physical therapy options for persons leaning to use and live with artificial limbs. In addition, there are discussions on how to use virtual reality to perform operations such as surgical operations and remote consultancy to increase the availability of doctors.
The third field that stands to benefit substantially from the technology is education. It is difficult to imagine all the potential applications of interactive multimodal applications in education. The projects reviewed above all require a high degree of engagement by the learners and therefore promises better understanding of concepts.
While most of them seem aimed at younger learners, mature students stand to benefit from learning in a 3-D learning environment. The application in modeling and simulation for technical subjects such as engineering and the teaching of concepts such as plate tectonics in geology, together with space exploration and medicine is promising.
Possible Research Directions Stimulated by the Paper
There are a number of possible research directions inspired by the project presented above. These include using the bubble as a basic unit for a collaborative problem-solving lesson and secondly as a health monitor. It also inspires the creation of a bubble that can take punches and transfer that input in the area of gaming. Increasingly, the world requires collaborative approaches to solve many of the problems that we face.
Problem-solving is a team activity, which requires teamwork by interdisciplinary professionals. The bubble presents an opportunity to begin training children on collaborative approaches to problem-solving. This means that the basic design discussed above can form the basis of an integrated platform designed to allow multiple users to enter information into a common task.
For instance, instead of one child drawing and transferring the information to the drawing canvas, several children all tackling a similar problem, may upload their information to the active task in order to complete it. The task will have to be predesigned while the data input will require adjustments to meet the specific needs of the task.
Secondly, the bubble presents an opportunity to monitor the health and environment of the child. An application like this will be practical in a hospital setting or at home, where there is a convalescing child or one with an ongoing medical condition where it is important to track the health of the child continually. The bubble can be both a toy and learning aid as well as a monitor that alerts care provides of any unwanted symptoms.
A simple design based on the one presented above but including adding eye-scanning software and vibration sensors such that each time the child squeezes out information to the canvas, it automatically uploads readings such as eye color and heart rate.
With networking, the convalescing child may actually be able to participate in class activities remotely by doing the same exercises as the rest of the class. Networking also makes possible connection to a hospital or doctor who receives alerts whenever the child’s condition deviates from the norm.
As an exercising aid, a bubble provides the opportunity to increase physical activity during a gaming session. Many games require a player to go through various stages as the game progresses. If there is an interjection during the game based on the level of play or on a regular interval basis, the player can be required to perform a physical activity before proceeding with the next level of play.
Two options for this include punching or kicking the bubble a number of times and using a preset amount of force in order to continue with the game, or by making the bubble the input device for such a game.
For instance, if the game in question involves fighting an opponent such as kickboxing, it would require physically hitting a bubble, which would have appropriate sensors to detect and transmit the sensations. In the process of playing the game, the player gets the satisfaction derived from gamming while at the same time gets to exercise.
In order to undertake a project in human-computer interaction, it important to use a combination of qualitative and quantitative research methods in order to achieve triangulation. Using both methods at different levels serves to increase the credibility of the results verified from two different approaches.
The generation of a great amount of information continues in the related areas, while a large number of software and hardware applications mediate the process. This makes it vital to understand the state of development of the technology in order to ensure research is current and adds useful information to the vast body of knowledge that already exists.
The key issues that require clarification include the nature of the problem, the benefits, and beneficiaries of a proposed solution, cost of implementation of the project, impacts of other fields on the project, impact of project to other fields and the available technologies and human resource.
Research methodologies fall in two major categories. These are quantitative methods and qualitative methods. Quantitative methods try to find out the degree accuracy of a stated hypothesis. It seeks to determine the numbers of people a particular issue affects and the extent of that effect.
It generally involves large-scale survey techniques, which are analyzed using statistical tools and it uses highly structured tools such as surveys, closed-ended questionnaires and structured observations as sources of data. Cohen, Manion and Morrison (2000) stated that, “typically quantitative methods require a degree of control and manipulation of phenomena” (p.119).
Qualitative research seeks out meanings. It is exploratory in nature. Qualitative research employs a number of different data collection methods. Its instruments are more flexible than those used in quantitative research are. Some of the methods used include observation, group discussions, narrations, and reflection.
It is important to include validation in qualitative research necessitated by the lack of hard figures to manipulate, as is the case with quantitative methods. Different tools are available for this task, including debriefing, negative case analysis and corroboration by an interviewer.
The areas of artificial augmented and virtual realities present a lot of potential for research development. There need for collaborative research is high because a lot of work is ongoing, and it is therefore very easy to apply duplicate efforts.
As Chen (2001) attests, “Human computer interaction is a multidisciplinary subject that involves information technology, computer science, psychology, library science, education, business and management, human factors, industrial engineering and ergonomics”(p. i) There is a key research opportunity driven by innovation based on current tools to meet emerging needs of the twenty-first century.
Chen, Q., (2001) Human computer interaction: issues and challenges. Pennsylvania: Idea Group Inc.
Cohen, L., Manion, L., & Morrison K. R. B.(2000) Research methods in education. New York: Routledge.
Grifantini, K. (2010) Malleable maps, artistic robots and bubble interfaces. In MIT Technology Review. Web.
Ryokai, K., Raffle, H. &Brooks, A.(2009). Proceedings from CHI 2009: Tangible Message Bubbles for Children’s communication and play . Boston: ACM.
Ryokai, K., Raffle, H. Horii, H. & Mann, Y. (2010) Proceedings from CHI 2010: Tangible Video Bubbles. Georgia: ACM.
Seba, N., Lew, M. S., &Huang, T. S. (2004). The state of the art in human computer interaction. Berlin: Springer-Verlag.