EDCI 337

 June 23rd, 2020

Alison Junger, Lauryn Lewis, 

Kirby Jarvis, and Sam Fry 

 

Remix of Chapter 9: The Modality Principle in Multimedia Learning.

 

What is the Modality Effect?

The modality effect refers to a cognitive load that appears when partially auditory and partially visual presentations of information work together to be more effective than when a single modality is used to present information. 

How does this theory affect teaching?

The modality effect is associated with the dual coding theory and the model of working memory and can be explained by the cognitive load theory. The working memory has a small capacity (it holds 5-9 pieces of information, only processes 2-4 and can only hold on to this information for about 20 seconds). Therefore, teaching methods are ineffective when they ignore this theory as it may overwhelm students’ cognitive loads.

How can we teach effectively with this in mind? 

There are three things to remember when designing instruction using the modality principle:

  1. The material being presented should be essential
  2. What is being shared is should not be redundant (both auditory and visually)
  3. Information is complex enough that techniques to reduce cognitive load are necessary

It should be noted that research has found that verbal information given orally in small fragments along with any other visual information is effective. On the other hand, long and complex verbal information is difficult to hold in working memory, therefore it should be shown visually.This will be discussed more in further sections of the remix (see transient information effect).

How does the working memory function?

Baddeley and Hitches three-component model of the working memory aligns with the idea that there are different processors for visuo-spatial and language based materials. The model contains a central executive which then divides into two components. The visuo-spatial sketchpad processes information in a visual and spatial form (such as colour and movement) whereas the phonological loop processes text and auditory material.(Low & Sweller p.229, 2014)

Dual Motive Presentation in the Classroom:

Research shows that dual mode presentation can result in increased performance on both memory tasks and monitoring tasks (Low & Sweller, p. 232, 2014). This is due to the fact that there are separate processors for visual and auditory information (see diagram above) in the working memory. Therefore, it should be theoretically possible to see greater retention of information when the modality effect is obtained. Namely, in the classroom, student academic performance can be enhanced by dual mode presentation techniques.  

How the Split Attention effect affects Student Learning:

The split attention effect is when information must be processed in two or more ways simultaneously in order to be understood. For example, in mathematics, the split attention effect occurs when students must refer to a text and a diagram at the same time in order to solve a problem. In other words, neither the text or the diagram alone convey enough information for the student to solve the problem. Therefore, by going back and forth between the diagram and the text, the brain is forced to memorize too much information at once and that the mental energy required to integrate the two different sources of information hinders learning (Low & Sweller, p. 235, 2014). Low and Sweller propose a more efficient way to integrate two sources of information, which is called the integrated format (p. 235, 2014). This is when the text and the image (or diagram) are combined and do not require students to flip back and forth between the two, which in turn, facilitates learning. This is crucial to keep in mind when creating visual powerpoints, textbooks, worksheets and assigning homework pages.

 

Another Take on the Modality Effect:

While many studies demonstrate the modality effect, there are some studies that contradict this theory (proving a reverse-modality effect). Tabbers and Van der Spoel’s study in 2011 concluded that when learning is self-paced and visual only instructions are provided there is a higher success rate. They argue that self-paced instruction allows one to transfer information from the working memory to the long-term memory more easily. An outside study, “The modality effect tested in children in a user‐paced multimedia environment” by M.J. Witteman and E. Segers conducted in 2010 also concluded that in self-paced settings there is a reverse modality effect. In this case, a sample of primary school students (around grade 5) actually learned better solely through visual instruction (text and images) when they were able to learn at their own pace (Witteman and Segers, p.1, 2010). This is important for educators to consider when teaching by distance (i.e. homeschooling/online schooling).

In most cases where studies failed to demonstrate the modality effect, it was because the audio or visual materials were far too complex and/lengthy and therefore overloaded the working memory. Leahy and Sweller believe that because spoken information is momentary, it can easily be forgotten and therefore there may be no advantage to portraying new complex information in this form (Low & Sweller, p. 239, 2014). This is called the transient information effect (Low & Sweller, p. 240, 2014). Overall this demonstrates that there are many variables to consider as an educator, and that the modality effect is not always found.

In an era of predominantly online instruction (due to the global pandemic), it is important to consider how this new way of teaching may cause a reversed modality effect. An article titled “The reverse modality effect: Examining student learning from interactive computer‐based instruction” by  Fethi A. Inan, Steven M. Crooks, Jongpil Cheon, Fatih Ari, Raymond Flores, Murat Kurucay and Dmitrii Paniukov in 2013 also proved a reverse modality effect. Inan et al. conducted a study that concluded that students who are taking courses online learn better through written instruction than through audio and text instruction. As mentioned above, this may also be due to the transient information effect (Low & Sweller, p. 240, 2014). 

How to achieve expansion and enhancement of the working memory?

 

Information is processed the best when the working memory is expanded to take in the total load of information. This can be supported through the dual coding theory and the split attention and interactivity effects (although there are some exceptions). When it comes to deciding how to present text, research has proven that a visual diagram combined with auditory text increases the working memory capacity because the information is split and relies on both parts of the memory for processing. This ties in with the dual coding theory, which activates both processors (which then strains, but does not overload the working memory). The result in an outside source by Wouters, Paas, and MerriĂ«nboer’s, states that “accompanying complex visual materials with written explanations may result in too much load on the visual channel and thus to inferior processing of the instructional materials” (Wouters, Paas & van MerriĂ«nboer, 2009), proving again that a visual diagram in the classroom with presented text may result in more successful student learning outcomes. Experienced learners may be an exception to this theory as they may learn best with a diagram alone. This also supports the notion of differentiated instruction within the classroom (Sousa and Tomilson, p.36 2018). The interactivity effect also plays a role in the expansion of the working memory, proving that high element interactivity materials alongside auditory/verbal instruction, enhances the modality effect and the ability to retain information (Low & Sweller p.238, 2014). Although, exceptions arise when either the auditory information or materials are redundant to one another. For materials with auditory instructions to be effective, they must follow the split attention principle (noted above). 

Conclusion:

Our remix has proven that the modality effect is highly dependent on the cognitive load of instructions. If instructions overload the working memory (such as long, highly complex or unfamiliar information) then learning is hindered (regardless of how the information is portrayed). These factors are very important for educators to understand when preparing lessons and materials. Because the modality effect is not entirely black and white, we can not simply focus on presenting information in two different forms, but we need to also consider how much information is being provided, what information is being provided and how it is being provided (i.e. the transient information effect). 

 

References: 

Inan, F.A., Crooks, S.M., Cheon, J., Ari, F., Flores, R., Kurucay, M. and Paniukov, D. (2015), Multimedia design principles. Br J Educ Technol, 46: 123-130. doi:10.1111/bjet.12129

Low, R., & Sweller, J. (2014). The Modality Principle in Multimedia Learning. The Cambridge Handbook Of Multimedia Learning, 2, 227 – 243.

Sousa, D., & Tomlinson, C. (2018). Differentiation and The Brain (2nd ed.). Bloomington, IN: Solution Tree Press.

Witteman, M. and Segers, E. (2010), The modality effect tested in children in a user‐paced multimedia environment. Journal of Computer Assisted Learning, 26: 132-142. doi:10.1111/j.1365-2729.2009.00335.x

Wouters, P., Paas, F., & van Merriënboer, J. (2009). Observational learning from animated models: Effects of modality and reflection on transfer. Contemporary Educational Psychology, 34(1), 1-8. doi: 10.1016/j.cedpsych.2008.03.001