By Cecile, Nancy and Mia
Photo Credit: Thomas Hawk via Compfight cc

Nowadays, teachers are quite familiar with teaching students to compose alphabetic texts (what is alphabetic? click here) in the digital environments, such as reports, academic essays, research papers, etc. However, they are often reluctant to introduce multiple modes in media into their writing class. One main reason is that teachers worry that the effects of computers and the vernacular speech expressions of multimodality will overshadow the main task of writing in their composition course.

But this is not true. The accessing of multimodality to the writing class has the potential to help students to create and interpet texts in today’s dynamic and increasingly technological world. As Selfe (2007) points out, composing print text has long been a multimodal process. In addition, ‘traditional’ media like film also offer a combination of visual and alphabetic information.

Sample: a multimodal composition

This is a student’s Multimodal Composition for English class at Full Sail (click here ), which provides us a direct image about how a multimodal composition can be conducted.

Challenges for Teachers

Selfe(2007) has identified some challenges that teachers may encounter when using multimodal composition in their teaching. The first challenge is both for the students and teachers. Choosing a composition topic can be a problematic issue for teachers. And in order for students to fulfill the task, they have to combine various kinds of meanings in an appropriate semiotic channel, including images, sounds, short videos, which can be rather a difficult mission. What’s more, without a hardcopy, for teachers and students, they are more familiar to correct and evaluate print-based compositions than the multimodal ones. Finally, in order to provide students with adequate support in their multimodal compositions, teachers have to be very familier with the relevant software and hardware. Otherwise, teachers’ lack of familiarity with relevent technology may create another obstacle.

Here are Some Guidelines for Teaching Audio and Video Composing:

  1. Introduce user-friendly software to students
  2. Act as a technical supporter during the task
  3. Ensure access to necessary equipment during the task
  4. Carefully present project models without limiting students’ creativity
  5. Provide clear rubrics to clarify the aim of the multimodal composing assignment and show how the assignment should be constructed


Selfe(2007) has considered a lot in the field of digital multimodal composition, but there are still a number of questions that need to be considered:

  1. How can teachers provide guidance for students as they go through the process of digital multimodal composition?
  2. Can multimodal composition be seen as an effective way to activate students’ motivation? To what extent do you agree or disagree?
  3. What advice would you give to teachers who are considering assigning multimodal composition?

45 thoughts on “A Guideline of Digital Multimodal Composition: Integrating Audio & Video into Writing

  1. After reading this article, I think the multimodal composition can be seen as an effective way to activate students’ motivation. Because students are engaged in a more challenging task which exercise their high autonomy to learn how to operate a specific technology tool to finish a project. What’s more important is that they have the chance to share their outcome online to worldwide readers.

    BTW: I like your guy’s portrait, very unitive!

    • Thank you, Joy. And yes, I agree with you that multimodal composition can be a very effective way to trigger students’ interests. They can use a variety of means to portray their work with the target language. And the composition is no longer only a pile of written language, but a hybrid of diverse kinds of output, including the oral language and the image.

    • Thank you, Joy. And yes, I agree with you that multimodal composition is a very effective way to increase students’ interests. A variety of means can be used to portray their work with the target language. And the composition is no longer only a pile of written words, but a hybrid of diverse kinds of output, including the oral language and the image.

    • Thank you Joy. Before doing the blog, I never thought about this kind of composition. If I were a student, I would be attracted by such a interesting way in improving second language proficiency. It is a way we can apply in the future teaching.

  2. Thank you for your helpful introduction, Mia, Nancy and Cecile, and I agree that adding video and audio to texts can be highly motivational for students. I do think there will need to be some in-class orientation for linking images, texts and videos together to send a particular message, though. I have my students read through a variety of multimodal texts before asking them to compose their own, and I think this helps.

    • Yes, I think playing multimodal compositions actually need far more preparations for both teachers and for students to do than the convential one. But no pain no gain, students will be able to practice more in such a kind of activity.

    • This seems to me to be essential – if the idea is to expand students’ ability to draw upon multiple semiotic resources, then they definitely need some kind of ‘input’ and instruction about the ways in which words, images and sound can be combined to make meaning.

  3. If we could make a good use of technology applied in multimodal composition, I think it could be a very effective way for the writers to illustrate what they virtually intend to express. The multimodal composotion showed in this article is a perfect case. By showing the images, readers receive the most direct visual information from the writer; By inserting the music, it helps the audiences get into the authentic environment the writer creates and easily generate resonance; By adding the words, readers can have a feeling of the writer’s emotions from his inner heart and be touched. However, what should the teacher notice here is that how to teach students to make a good multimodal composition by combining those elements (e.g.: images, sounds, videos, words, etc) properly and exactly reflect on their thinking.

    • I like your point. I agree with the affordance of the application of multi-modal composition. And for the last question about how to teach SS to make a good composition by combining those elements, as you mentioned, I think teachers can direct SS to focus on certain functions of each semiotic means instead of all them all. And teachers should let ss know what part is the central in completing the composition, that is , their literacy. And to make it clear, teachers can give specific assessment rubrics to make ss focus on the point.

  4. I believe that multimodal composition will be prevailing in teaching in the future, and I think using PPT is a good start for this kind of writing. We usually integrate texts, hypertexts, pictures, videos together in a PPT to present our ideas. Moreover, it can be said that we rely on using PPT in teaching and learning today. As for me, sometimes I feel that I can not convey my ideas clearly in a presentation without PPTs. This indicates that multimodal representation has actually become an important part in classroom. We are quite familiar with this kind of writing that it will be not difficult for us to write an academic paper with multimodal representation. What we need are some practices and time to get used to it. But technical instructions and training are still necessary especially at the very beginning. Thus, it is better that teachers learn and master the techniques before assigning multimodal composition.

    • Thank you, Anika. PPT is really a good start for students who have never learn about the multimodal composition. And I think apart from the necessary techniques to manipulate the digital device, teachers should also help their students to learn about the affordance of these multimodal means. Because only if students know something about the characteristic of these multimodal device can they adjust their language to better make use of the multimodal tools.

  5. Students are inspired to become creative during the process of multimodal composition. Creativity can be shown in the various different arrangements of multimodal elements. In addition to the writing with given topics, teachers can provide students with the exactly same multimodal materials, and allow students to “write” for themselves, and then compare the different outcomes which may cover different topics and use various genres. In this case, the materials should be much more general, leaving students enough space to generate creativity and compose in different ways.

    • Yes, Clover. I cannot agree more. As far as I am concerned, creativity should be encouraged in any forms of Education. Through the application of multi-model approach in language education, learners are able to learn the world, culture and so many other things than the language itself. Therefore, language education can be more integrated for both teachers and learners, and of course, many learners will find learning languages more appealing.

  6. I think students will benefit a lot from multimodal composition teaching, for multimodality is the new literacy that learners use everyday. Also, through multimodal composition teaching, students can learn writing skills, new genre, collaboration, etc.

    • Yes. multimodal composition actually trains students to master more than basic language skills. Students also need to learn how to edit pictures, videos and so on. And these skills are actually very useful in daily life as well.

  7. Younger generations are living in the world of digital age. They are familiar with new technologies even before they go to school. Teachers have to apply multimodal composition in class because that’s the trend of the era. I think it’s like the video we saw several weeks ago. Students have got most of the idea what is going on in class. Teachers only need to tell them rules and monitoring the whole process. That requires for teachers’ proficiency. So as teachers, we should access more to new technologies, catch up the trend.

    • Lily, I agree with you. Just as what I said above, multimodal composition will sooner or later become prevailing in classroom teaching. That is the trend. On one hand, I am sure that the new generation can use digital technologies very well. On the other hand, I think helpful instructions to some technical problems are still necessary. Maybe the school can open some courses teaching students how to use digital media.

    • Interesting observations. There is a tendency to treat the ‘digital natives’ as though they are already fluent in technological practices. However, often this is not the case and instruction and support is necessary.

    • Nice comment. I like the point that “younger generations are living in the world of digital age”. They already gain lots of technical skills. So teachers are suggested to provide some instructions to combine the language learning and digital world together.

  8. Thank you for sharing. For the first question, I think students can be divided into groups and all the groups can share their ideas or ask questions on a platform. It’s easier for teachers to help students go through his kind of collaborative work. I also think that multimodal composition is motivating if the topic is interesting with appropriate degree of difficulity.

    • Thank you, Cassie. I like your idea about “appropriate degree of difficulty”. I think the control over the level of difficulty means a lot in the multimodal composition classroom. Since the multimodality has already given students some challenges, the theme or title of the composition itself shouldn’t be too difficult. Otherwise students may lose their passion.

  9. First of all, I really love you three’s avatars!!!!!!
    In my opinion, multimodal composition is a good way to improve students’ writing skills since they can write more creatively and freely. Before designing the task, teachers should first do analysis on students’ needs as well as learning environment. I mean the task should fit students’ learning objectives and they have the ability and equipment to complete the task.
    One of my friend is an English teacher in a secondary school in Guangzhou. In the winter vocation, he asked his students, who are in senior three, to choose a university in the city that they are really interested or want to study in in the future. Students should first go around the campus of that university, take pictures and then shared with classmates in any forms they want, powerpoint, video, picture presentation…. they had to describe that university, what it looks like, why you like it, what major do you want to choose…and these compositions will be stuck on the walls. Finally, they wrote down what they said in words and stuck these compositions on the wall.
    I think this is a very good example of multimodal composition. Students use what they have learned to write, and at the same time, they are creative and highly-motivated.

  10. I remember in 2008, when I was in high school, a terrible thing happened in China. That is the big earthquake in Wenchuan, Sichuan Province. My teacher wrote an article himself about the earthquake and read it to us while using PowerPoint to show text, pictures and play music, video. Everybody in the class was deeply impressed and crying so hard when when we watching this. By using this example I want to say that the multimodal composition is highly attractive. Both the composer and the reader will be much more involved in such a process. They gain passion and motivation from project like this. I agree with the difficulty in choosing right topics. Because it’s better to attract students’ interests. They need to feel the desire to express and they want to express in a fantastic way. Maybe giving them autonomy to choose what they want to write is good (my teacher tried this in college, we all put great efforts in it) but the assessment will be another problem if we do so.

    • Thank you, Sherry. Yes, I admit that the assessment of this kind of multimodal composition can be a little bit complicated. But if the teachers give the rubrics to the students in advance, everything will be much easier to be done. With the universal standards, students can complete their composition following the criteria of evaluation. Then the quality of their composition can be guaranteed.

    • Actually my teacher did the same thing! I think that’s one shining point about multimodality composition. It may last longer than traditional writing elements by use of images and sounds.

    • Thank you for your sharing. I think multimodal composition can not only just being a specific assignment style but also can be used in language teaching process, only power point s sometimes too boring, right? Maybe teacher’s can also use multimodal tools in there teaching process.

  11. I like the video you choose to introduce multimodality composition, that’s definitely motivating not only for students but also for teachers. For students, in addition to writing, speaking is practiced as well as video making skills. For teachers, evaluating students’ writing can be very exhausting sometimes, but with the video, things can be very different in a good way. But I am afraid that not every topic can be “written” in this fantastic way. Teachers may need to choose some topics that are close to students’ real life.

    • Thank you, Tracy. I also hold that the topic of these multimodal compositions should be selected carefully. Teachers shall choose topics that can bring into full play the characteristics of multimodality.

  12. Thank you so much for your article. First of all, when I was in high school, I didn’t have the chance to experience this multimodal learning approach. when my teacher taught the new language points, I just felt so bored, and I had no interests in learning. While I think this approach gives the learners strong motivation, let them have interests in learning, which is so efficient. Also, for teachers, they would feel stress when they have to find some interesting ways to teach four skills of language, this multimodal approach really help the teachers.

    • Yes. In my opinion, pictures and videos are more vivid and impressive for students to remember than all the boring words. Multimodal approach are beneficial for learners to link language skills to daily use.

    • Motivation is the most important predictor for learning a language successfully. Traditional teaching methods are quite monotonous that students easily feel bored in the process of learning. However, new technologies nowadays provide a lot of possibility to language teacihng. Like this one, multimodal composition will increase students’ great interest in learning, which ensures students to keep on learning to a large extent.

  13. Based on the video above, I took the teacher is try to make the student to do a creative writing combing the images to a sole story. From my point of view, it is a brilliant approach since it not only provide more opportunities for students to practice writing skills but also in the task students can create with less limitation than other forms of writing. But as mentioned, teachers should provide proper guideline for students for that writing task. For example, leave enough space for students to create by giving a lager scale of topic and what’s more, the technical support is significant.

  14. I love all your opinions, and I also want to add that skills learned through the multimodal composition are not only useful for students at present, but also good skills in working place in the future. Most of the time in the work place, people need to know those skills to accomplish their own work. Therefore, it is also a good chance to learn new skills.

    • I agree with you Gloria. New Literacies skills exist everywhere. It is necessary and beneficial for people to learn these skills to keep pace with the world.

  15. I think it is difficult in some cases to introduce multimodal learning into the classroom setting. From my experience in local HK schools, the schools themselves are just not equipped with the technology to conduct the lesson in such a way. Aside from a computer at the teacher’s desk that often has issues and on an Internet bandwidth that is quite slow, it is hard to even just use that one computer. Some of the lessons and ways of teaching some of these lessons are great for a computer lab but where can we get that setting for an English lesson? Not only that, how do we keep some regular lessons in that setting for the whole school year? I guess they could use their imaginations and act like they are on a computer.

  16. I think motimodality is the product of the morden society. We like it when we take class, our teacher use pictures, videos, etc.. It can greatly improve the students’ motivation of studying. Also, the involvement of students will be promoted in class as well as after class. But sometimes the time can be a issue because it requires more time than normal class. But I really think it is great to use this kind of approach in learning.

    • Yeah, Emma, I think you made a very good point about the time management. Time is really a big problem in this kind of class. Indeed students are more involved in it, but more often than not, they tend to leave the work till next class because the lack of time. Students just cannot finish the multimodal work in a short time, which leaves the teaching tasks hard to be consistent from time to time.

  17. Though students themselves may be familiar with digital technology, they may still possess the ideology that language learning should be carried out in conventional ways. I have witnessed students who just focused on taking language points the teacher taught and the related exercise in test paper. Any kind of interesting, creative, less language points focused activities may be meaningless to them. This phenomenon is not rare in senior high schools, especially for those students who are good at coursework. They would insist on teacher teach something “useful” and “important”. To apply the multimodality in class, teachers should first tell the student the importune of this kind of learning approach and shifted students’ ideology and habitus of learning through grammar, vocabulary in written discourse.

    • Hello Fiona. Your comment reminds me of what we learned in SLA. Sometimes students’ beliefs are different from teachers’ beliefs, which may generate a lot of problems. In your case, students do not understand why they have to learn and use technology to finish their job, so that they will ignore this part and what’s worse, generate negative attitude towards the teacher and the lessons.

      You also give a good solution to this problem. Teachers should explain the reasons to students before the task. But sometimes it is not easy to shift their ideology, because the form-based teaching and learning limites their imagination. For short-term effect, teachers can make technologies included in the assessment part. In this way, maybe students will pay more attention to the technology.

  18. I agree with you. The form and content of a class should be given equal importance in language learning .Potential obestacles of implementation should also be taken into consideration during the planing of the class.Language points should be highlighted and emphasized repeatedly especially when learners are young children.,

  19. I think multimodal compositions are good way to increase motivation to read in a foreign language classroom. My experience has been that texts which incorporate images and videos (possibly audio as well) offer additional semiotic means of communication that aids in comprehending the text. A picture communicates more meaning than text can and does not rely on language for its meaning (although one’s culture may influence one’s interpretation), and this can lessen the ‘cognitive load’ of reading a text in a foreign language.

Comments are closed.