A Multimodal Learning Approach for Translating Live Lectures into MOOCs Materials
學年 112
學期 2
發表日期 2024-07-09
作品名稱 A Multimodal Learning Approach for Translating Live Lectures into MOOCs Materials
作品名稱（其他語言）
著者 Tzu-Chia Huang; Chih-Yuan Chang; Hung-I Tsai; Han-Si Tao
作品所屬單位
出版者
會議名稱 IEEE ICCE-TW
會議地點 Taichung; Taiwan
摘要 This paper introduces an AI-based solution for the automatic generation of MOOCs, aiming to efficiently create highly realistic instructional videos while ensuring high-quality content. The generated content strives to keep content accuracy, video fluidity, and vivacity. This paper employs a multimodal to understand text, images, and sound simultaneously, enhancing the accuracy and realism of video generation. The process involves three stages: First, the preprocessing stage employs OpenAI's Whisper for audio-to-text conversion, supplemented by Fuzzy Wuzzy and Large Language Models (LLMs) to enhance content accuracy and detect thematic sections. In the second stage, speaker motion prediction begins with skeleton tags. Based on these labels, the speaker’s motion can be classified into different categories. Subsequently, a multimodal, including BERT and CNN, further extracts features from text and voice diagrams, respectively. Based on these features, the multimodal can learn the speaker’s motion categories through the skeleton labels. As a result, the multimodal can predict the classes of the speaker’s motions. The final stage generates MOOCs audiovisuals, converting text into subtitles using LLMs and predicting the speaker’s motions. Finally, the wellknown tool is used to ensure accurate voice and lip synchronization. Based on the mentioned approaches, the proposed mechanism guarantees seamless alignment and consistency in the video elements, thereby ensuring the generated MOOCs can be realistic and more recent.
關鍵字 Generative MOOCs; Instructional videos; multimodal; Skeleton-based motion classification; Extractive summarization
語言 zh_TW
收錄於
會議性質 國內
校內研討會地點 無
研討會時間 20240709~20240711
通訊作者
國別 TWN
公開徵稿
出版型式
出處
相關連結

機構典藏連結 ( http://tkuir.lib.tku.edu.tw:8080/dspace/handle/987654321/126358 )

SDGS 尊嚴就業與經濟發展,產業創新與基礎設施