Journal of Research in Rehabilitation Sciences
Converting Human Movements into a Conceptual Painting based on Music Using a Motion Capture System: Action Research
Document Type : Original Articles
Authors
1 MSc Student in Computers Arts, School of Multimedia, Tabriz Islamic Art University, Tabriz, Iran
2 Assistant Professor, School of Multimedia, Tabriz Islamic Art University, Tabriz, Iran
Abstract
Introduction: Motion capturing is a concept which has been taken into consideration in recent decades and it has been used for various purposes like improving the performance of patients with motor problems and increasing motivation and creativity of users. It can also be used in film industry both in two- and three-dimensional animations. Research has shown that this system may be used to enhance creativity and intellect. The aim of this study is to use the Vicon motion capturing device to convert creative human movements into a conceptual painting with the help of music.Materials and Methods: For cultural adaption of the system, the first chapter of “HAFTVAD” music album called “The Introduction” composed by Masoud Nekoyi and the story of Rostam and Sohrab from Shahnameh, written by the Persian poet Ferdowsi, were chosen as the musical groundwork and the scenario for this survey. A human model was trained to perform movements in accordance with the musical and fictional workarounds of the study; the model was also allowed to use self-created styles in movements in the final performance. The segmental movements of the model body were identified in the Vicon Vero system by wearing clothes equipped with reflective markers. This information was transmitted into the Unity engine, and without modeling the body's segments, different colors and brushes were defined to illustrate the path of the movements of the hands, feet, head, and heart.Results: By defining various brushes in different colors, human movements were converted into a conceptual painting in accordance with the music.Conclusion: This study discussed a creative process, entitled Motion Paint, which was implemented in Iran for the first time. This method can be used to make the process of exercise therapy more attractive, especially in chronic motor problems.
Keywords
- Sears A, Jacko JA. The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, (Human Factors and Ergonomics). 2nd ed. Boca Raton, FL: CRC Press; 2007.
- Annum GY. Digital painting evolution: A multimedia technological platform for expressivity in fine art painting. Journal of Fine and Studio Art 2014; 4(1): 1-8.
- Suwetha R, Subedha V, Kalaichelvi T, Hemalatha S. Motion Capture using 3D. International Journal for Research in Applied Science and Engineering Technology 2017; 5(4): 1220-7.
- Cheung KM, Kanade T, Bouguet J, Holler M. A real time system for robust 3D voxel reconstruction of human motions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat.No.PR00662); 2000 Jun 15; Hilton Head Island, SC, USA.
- Bevilacqua Fdr, Naugle L, Valverde I. Virtual dance and music environment using motion capture. Proceeding of the IEEE Multimedia Technology and Applications Conference (MTAC); 2001 Nov 8-11; Irvine, CA, USA.
- Bevilacqua Fdr, Ridenour J, Cuccia D. 3D motion capture data: Motion analysis and mapping to music. Proceedings of the 6th Distributed Memory Computing Conference; 1991 Apr 28- May 1; Portland, OR, USA.
- Yang Z, Yu B, Wu W, Diankov R, Bajcsy R. Collaborative dancing in tele-immersive environment. Proceedings of the 14th ACM International Conference on Multimedia; 2006 Oct 23-27; Santa Barbara, CA, USA.
- Kapur A, Tzanetakis G, Virji-Babul N, Wang G, Cook PR. A framework for sonification of vicon motion capture data. Proceedings of the 8th International Conference on Digital Audio Effects (DAFX-05); 2005 Sep 20-22; Madrid, Spain.
- Vicon Visualization Tool [Online]. [cited 2020]; Available from: URL: https://www.vicon.com/visualization/
- Meisner J. dance(with)in [Online]. [cited 2016 Nov 15]; Available from: URL: https://jensmeisner.net/dancewithin/
)