Keywords
second-order meta-analysis
meta-analysis
learning
school children
educational outcomes
mobile learning
digital behavior
Abstract
Introduction. The use of mobile devices by children and adolescents is a common practice at the moment. Although mobile learning has some benefits, the results of the study on its effectiveness are contradictory. The study described in this paper is one of the first carried out to summarize the results of meta-analyses assessing the effects of using mobile devices by school children in the educational context. Methods. The study aimed to identify the effects of using mobile devices in learning based on second-order meta-analysis procedures. Twenty-nine meta-analyses conducted between 2014 and 2023 were analyzed, with minimal overlap between primary studies. A systematic search for sources and their assessment in accordance with the meta-review protocol, an analysis of coincidences in studies included in primary meta-analyses, an assessment of the presence of publication bias, and an analysis of the influence of categorical moderators were conducted. Results. The use of mobile devices by students has an average impact on their educational outcomes (g = 0.654 (95% CI: 0.578–0.73)). A significant result was found when assessing the heterogeneity of mean effect sizes (t2 = 0.042, Q = 277.255, p < 0.001; I2 = 86.95 %). The moderator analysis showed a significant impact of the type of educational outcome, subject area of study, type of publication and location of primary research. Conflicting results are found when mean effect sizes are analyzed across different levels of education. Discussion. The average effect size obtained during the analysis is characterized by a high degree of stability at different periods of digitalization of education. The use of mobile devices by school children can have a dual impact on their learning activities, depending on the inclusion of adults in this process. Problematic digital behavior is associated with worsened educational outcomes. The use of gadgets for educational purposes, on the contrary, helps to increase learning efficiency compared to its traditional forms. The meta-review provides further directions for research on the effects of the use of mobile devices by school children in their educational activities and can help develop digital culture education programs.
References
Afikah, A., Astuti, S. R. D., Suyanta, S., Jumadi, J., & Rohaeti, E. (2022). Mobile Learning in Science Education to Improve Higher-Order Thinking Skills (HOTS) and Communication Skills: A Systematic Review. International Journal of Advanced Computer Science and Applications, 13(7). https://doi.org/10.14569/IJACSA.2022.0130782
Akçay, A. O., Karahan, E., & Bozan, M. A. (2021). The Effect of Using Technology in Primary School Math Teaching on Students’ Academic Achievement: A Meta-Analysis Study. FIRE: Forum for International Research in Education, 7(2), 1–21. https://doi.org/10.32865/fire202172231
Alfadil, M. (2020). Effectiveness of virtual reality game in foreign language vocabulary acquisition. Computers & Education, 153, 103893. https://doi.org/10.1016/j.compedu.2020.103893
Avdeeva, E. A., & Kornilova, O. A. (2022). Influence of digital environment on the cognitive functions of schoolchildren and students. Cardiovascular Therapy and Prevention, 21(S3), 3331. https://doi.org/10.15829/1728-8800-2022-3331 (in Russ.)
Barr, R. (2019). Growing up in the digital age: Early learning and family media ecology. Current Directions in Psychological Science, 28(4), 341–346. https://doi.org/10.1177/0963721419838245 PMID:31423053
Bezgodova, S., Miklyaeva, A., & Nikolaeva, E. (2020). Computer vs smartphone: How do pupils complete educational tasks that involve searching for information on the internet. In SEUR Workshop Proceedings, 2630, 52–62.
Biber, S. K., Biber, M., & Erbay, H. N. (2022). Teachers’ perceptions on technology-assisted mathematics teaching and the interactive activities. Education and Information Technologies, 27(5), 6913–6945. https://doi.org/10.1007/s10639-022-10898-9
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021). Introduction to meta-analysis. John Wiley & Sons.
Borozdina, N. (2022). Mediascope CROSS WEB. Internet audience in 2022. https://mediascope.net/upload/iblock/3d8/qrlhud7t7dxyzw1rhtzxg3rwk8deg7uk/2022_%D0%98%D0%9D%D0%A2%D0%95%D0%A0%D0%9D%D0%95%D0%A2. pdf (in Russ.)
Čevajka, J., & Velmovská, K. (2022). The use of mobile technologies in teaching from the perspective of the Slovak physics teachers. In Cieśla, P. (Ed.) Active Science Education. (pp. 7-19). University of the National Education Commission. https://doi.org/10.24917/9788380848139.1
Chang, C. J., Liu, C. C., Wen, C. T., Tseng, L. W., Chang, H. Y., Chang, M. H., Chiang, S. H. F., Hwang, F. K., & Yang, C. W. (2020). The impact of light-weight inquiry with computer simulations on science learning in classrooms. Computers & Education, 146, 103770. https://doi.org/10.1016/j.compedu.2019.103770
Chau, K., Bhattacherjee, A., Senapati, A., Guillemin, F., & Chau, N. (2022). Association between screen time and cumulating school, behavior, and mental health difficulties in early adolescents: A population-based study. Psychiatry Research, https://doi.org/10.1016/j.psychres.2022.114467
Chen, M. L. (2022). The Impact of Mobile Learning on the Effectiveness of English Teaching and Learning – A Meta-Analysis. IEEE Access, 10, 38324-38334. https://doi.org/10.1109/ACCESS.2022.3165017
Chen, Z., Chen, W., Jia, J., & An, H. (2020). The effects of using mobile devices on language learning: A meta-analysis. Educational Technology Research and Development, 68. https://doi.org/10.1007/s11423-020-09801-5
Chistova, Yu. S., & Krotkova, I. N. (2018). Mobile apps as a resource for motivating schoolchildren when learning English. Scientific Search, (3), 16–19. (in Russ.).
Cho, K., Lee, S., Joo, M. H., & Becker, B. J. (2018). The effects of using mobile devices on student achievement in language learning: A meta-analysis. Education Sciences, 8(3), 105. https://doi.org/10.3390/educsci8030105
Cooper, H., & Koenka, A.C. (2012) The overview of reviews: unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist, 67(6), 446-462. https://doi.org/10.1037/a0027119
Ericsson Mobility Visualizer – Mobility Report. (2023, February 23). Telefonaktiebolaget LM Ericsson. https://d110erj175o600.cloudfront.net/wp-content/uploads/2022/11/29180338/Ericsson-Mobility-Report-November-2022.pdf
Fadel, C., Bialik, M., & Trilling, B. (2018). Four-dimensional education: The competencies learners need to succeed. Tochka. (in Russ.).
Feng, Y., Liao, Y., & Ren, Y. (2018). Effects of M-Learning on Students’ Learning Outcome: A Meta-analysis. In L. Deng, W. W. K. Ma, & C. W. R. Fong (Eds.), New Media for Educational Change (pp. 115–123). Springer. https://doi.org/10.1007/978-981-10-8896-4_10
Garzón, J., Lampropoulos, G., & Burgos, D. (2023). Effects of Mobile Learning in English Language Learning: A Meta-Analysis and Research Synthesis. Electronics, 12(7), 1595. https://doi.org/10.3390/electronics12071595
Gordeeva, N. O. (2018). The use of mobile technologies in education: A meta-analysis of Russian studies. Modern problems of science and education (digital scientific journal). https://science-education.ru/ru/article/view?id=27704 (in Russ.)
Goshin, M. E., Grigor'iev, D. S., & Mertsalova, T. A. (2019). Types of parental involvement in education and academic performance of schoolchildren. Voprosy Psikhologii, 2, 29–44. (in Russ.).
Güler, M., Bütüner, S. Ö., Danişman, Ş., & Gürsoy, K. (2022). A meta-analysis of the impact of mobile learning on mathematics achievement. Education and Information Technologies, 27, 1725–1745. https://doi.org/10.1007/s10639-021-10640-x
Güzeller, C. O., & Üstünel, F. (2016). Effects of Mobile Learning on Academic Achievement: A Meta Analysis. Journal of Social Sciences/Sosyal Bilimler Dergisi, 8(23), 528–561.
Hennessy, E. A., & Johnson, B. T. (2020). Examining overlap of included studies in meta‐reviews: Guidance for using the corrected covered area index. Research synthesis methods, 11(1), 134–145. https://doi.org/10.1002/jrsm.1390
Jin, W., & Sabio, C. J. (2018). Potential use of mobile devices in selected public senior high schools in the city of Manila Philippines. International Journal of Learning, Teaching and Educational Research, 17(4), 102–114. https://doi.org/10.26803/ijlter.17.4.7
Kapina, A. A. (2020). Modern solutions to the problem of managing distance learning using mobile applications. Pedagogical Bulletin, 15, 24–26. (in Russ.).
Kärchner, H., Trautner, M., Willeke, S., & Schwinger, M. (2022). How handheld use is connected to learning-related factors and academic achievement: Meta-analysis and research synthesis. Computers and Education Open, 3, 100116. https://doi.org/10.1016/j.caeo.2022.100116
Kates, A. W., Wu, H., & Coryn, C. L. S. (2018). The effects of mobile phone use on academic performance: A meta-analysis. Computers & Education, 127, 107–112. https://doi.org/10.1016/j.compedu.2018.08.012
Keegan, D. (2000). From Distance Learning To E-learning To Mobile Learning. Open Education Research, 5, 211–217.
Kornilov, S. A., & Kornilova, T. V. (2010). Meta-analytic studies in psychology. Psychological Journal, 31(6), 5–17. (in Russ.).
Lee, Y. S., Sung, Y. T., Chang, K. E., Liu, T. C., & Chen, W. C. (2014). A meta-analysis of the effects of learning languages with mobile devices. In New Horizons in Web Based Learning: ICWL 2014 International Workshops, SPeL, PRASAE, IWMPL, OBIE, and KMEL, FET, Tallinn, Estonia, August 14-17, 2014, Revised Selected Papers 13 (pp. 106–113). Springer International Publishing. https://doi.org/10.1007/978-3-319-13296-9_12
Lei, H., Chiu, M. M., Wang, D., Wang, C., & Xie, T. (2022). Effects of Game-Based Learning on Students’ Achievement in Science: A Meta-Analysis. Journal of Educational Computing Research, 60(6), 1373–1398. https://doi.org/10.1177/07356331211064543
Li, S., Xu, M., Zhang, Y., & Wang, X. (2023). The more academic burnout students got, the more problematic mobile phone use they suffered? A meta-analysis of mainland Chinese adolescents and young adults. Frontiers in Psychology, 13, 1084424. https://doi.org/10.3389/fpsyg.2022.1084424
Liao, Y., Chen, C.-H., & Yang, Y.-C. (2020). A meta-analysis of the effects of mobile learning on students’ academic achievement in Taiwan. Contemporary Educational Research Quarterly, 28, 67–102. https://doi.org/10.6151/CERQ.202009_28(3).0003
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., ... & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Annals of internal medicine, 151(4), W-65. https://doi.org/10.7326/0003-4819-151-4-200908180-00136
Liu, Q., & Wu, J. (2023). What children learn in a digital home: the complex influence of parental mediation and smartphone interference. Education and Information Technologies, 1–19. https://doi.org/10.1007/s10639-023-12071-2
Malkova, E. E., & Naumova, A. A. (2012). Adolescence as a clinical and psychological phenomenon in science and culture. Medical psychology in Russia, (6), 17–17. (in Russ.).
McDaniel, B. T., & Radesky, J. S. (2018). Technoference: Longitudinal associations between parent technology use, parenting stress, and child behavior problems. Pediatric research, 84(2), 210–218. https://doi.org/10.1038/s41390-018-0052-6
Mihaylova, M., Gorin, S., Reber, T. P., & Rothen, N. (2022). A Meta-Analysis on Mobile-Assisted Language Learning Applications: Benefits and Risks. Psychologica Belgica, 62(1), 252. https://doi.org/10.5334/pb.1146
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
Mukhametzyanov, I. Sh. (2019). Smartphone at school: “To be or not to be, that is the question.” Informatics in school, 7(150), 52–56. https://doi.org/10.32517/2221-1993-2019-18-7-52-56 (in Russ.)
Mustafa Kamal, N. N., Mohd Adnan, A. H., Yusof, A. A., Ahmad, M. K., & Mohd Kamal, M. A. (2019, January). Immersive interactive educational experiences–adopting Education 5.0, Industry 4.0 learning technologies for Malaysian Universities. In Proceedings of the International Invention, Innovative & Creative (InIIC) Conference, Series (pp. 190–196).
Mutambara, D., & Bayaga, A. (2021). Determinants of mobile learning acceptance for STEM education in rural areas. Computers & Education, 160, 104010. https://doi.org/10.1016/j.compedu.2020.104010
Nakano, M. (2022). Artificial intelligence and robotic process automation for accounting and auditing in society 5.0. Journal of Social Science, 89, 51–61. https://doi.org/10.34577/00005073
Novikov, M. Yu., & Starichenko, B. E. (2020). Construction of the school informatics course based on mobile and cloud technologies. Informatics in school, 1(154), 40–54. https://doi.org/10.32517/2221-1993-2020-19-1-40-54 (in Russ.)
Novikova, I. I., Zubtsovskaya, N. A., Romanenko, S. P., Kondrashchenko, & A. I., Lobkis, M. A. (2020). Effects of mobile phones on children’s and adolescents’ health. The Science of the Person: Humanitarian Researches, 2, 95–103. https://doi.org/10.17238/issn1998-5320.2020.14.2.16 (in Russ.)
Okumuş Dağdeler, K. (2023). A systematic review of Mobile-Assisted Vocabulary Learning research. Smart Learning Environments, 10(1), 19. https://doi.org/10.1186/s40561-023-00235-z
Öztürk, B., Kaya, M., & Demir, M. (2022) Does inquiry-based learning model improve learning outcomes? A second-order meta-analysis. Journal of Pedagogical Research, 6(4), 201–216. https://doi.org/10.33902/JPR.202217481
Panov, V. I., Borisenko, N. A., Mironova, K. V., & Shishkova, S. V. (2021). Behavior of adolescents in the digital educational environment: Definition of concepts and formulation of the problem. Izvestiya of Saratov University. New Series. Series: Educational Acmeology. Developmental Psychology, 3(39), 188–196. https://doi.org/10.18500/2304-9790-2021-10-3-188-196 (in Russ.)
Petersen‐Brown, S. M., Henze, E. E., Klingbeil, D. A., Reynolds, J. L., Weber, R. C., & Codding, R. S. (2019). The use of touch devices for enhancing academic achievement: A meta‐analysis. Psychology in the Schools, 56(7), 1187–1206. https://doi.org/10.1002/pits.22225
Pieper, D., Antoine, S.-L., Mathes, T., Neugebauer, E. A. M., & Eikermann, M. (2014). Systematic review finds overlapping reviews were not mentioned in every other overview. Journal of Clinical Epidemiology, 67(4), 368–375. https://doi.org/10.1016/j.jclinepi.2013.11.00
Romadiah, H., Dayurni, P., & Fajari, L. E. W. (2022). Meta-Analysis Study: The Effect of Android-Based Learning Media on Student Learning Outcomes. International Journal of Asian Education, 3(4), 253–263. https://doi.org/10.46966/ijae.v3i4.300
Sahin, D., & Yilmaz, R. M. (2020). The effect of augmented reality technology on middle school students' achievements and attitudes towards science education. Computers & Education, 144, 103710. https://doi.org/10.1016/j.compedu.2019.103710
Schmidt, F. L., & Oh, I. S. (2013). Methods for second-order meta-analysis and illustrative applications. Organizational Behavior and Human Decision Processes, 121(2), 204–218. https://doi.org/10.1016/j.obhdp.2013.03.002
Selwyn, N., Hillman, T., Eynon, R., Ferreira, G., Knox, J., Macgilchrist, F., & Sancho-Gil, J. M. (2020). What’s next for Ed-Tech? Critical hopes and concerns for the 2020s. Learning, Media and Technology, 45(1), 1–6. https://doi.org/10.1080/17439884.2020.1694945
Shi, L., & Kopcha, T. J. (2022). Moderator effects of mobile users’ pedagogical role on science learning: A meta-analysis. British Journal of Educational Technology, 53(6), 1605–1625. https://doi.org/10.1111/bjet.13210
Soldatova, G. U., Rasskazova, E. I., Vishneva, A. E., Teslavskaya, O. I., & Chigarkova, S. V. (2022). Born digital: Family context and cognitive development. Akropol. (in Russ.).
Spasskaya, E. B., & Proekt, Yu. L. (2023). Mobile devices of students in the school learning environment through lens of teacher assessment. Perspectives on Science and Education, 3(63), 603–618. https://doi.org/10.32744/pse.2023.3.36 (in Russ.)
Sunday, O. J., Adesope, O. O., & Maarhuis, P. L. (2021). The effects of smartphone addiction on learning: A meta-analysis. Computers in Human Behavior Reports, 4, 100114. https://doi.org/10.1016/j.chbr.2021.100114
Sung, Y.-T., Chang, K.-E., & Yang, J.-M. (2015). How effective are mobile devices for language learning? A meta-analysis. Educational Research Review, 16(1), 68–84. https://doi.org/10.1016/j.edurev.2015.09.001
Sung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252–275. https://doi.org/10.1016/j.compedu.2015.11.008
Sung, Y.-T., Yang, J.-M., & Lee, H.-Y. (2017). The Effects of Mobile-Computer-Supported Collaborative Learning: Meta-Analysis and Critical Synthesis. Review of Educational Research, 87(4), 768–805. https://doi.org/10.3102/0034654317704307
Talan, T. (2020). The Effect of Mobile Learning on Learning Performance: A Meta-Analysis Study. Educational Sciences: Theory & Practice, 20(1), 1. https://doi.org/10.12738/jestp.2020.1.006
Talan, T., Y. Doğan, & Batdı V. (2020). Efficiency of Digital and Non-Digital Educational Games: A Comparative Meta-Analysis and a Meta-Thematic Analysis. Journal of Research on Technology in Education, 52(4), 474–514. https://doi.org/10.1080/15391523.2020.1743798
Tamim, R. M., Borokhovski, E., Pickup, D., Bernard, R. M., & El Saadi, L. (2015). Tablets for teaching and learning: A systematic review and meta-analysis. Commonwealth of Learning. Retrieved from: https://oasis.col.org/server/api/core/bitstreams/678099a2-4d78-4fc3-a6b8-3f9e81f51c7e/content
Tingir, S., Cavlazoglu, B., Caliskan, O., Koklu, O., & Intepe‐Tingir, S. (2017). Effects of mobile devices on K–12 students' achievement: A meta‐analysis. Journal of Computer Assisted Learning, 33(4), 355–369. https://doi.org/10.1111/jcal.12184
Troll, E. S., Friese, M., & Loschelder, D. D. (2020). How students' self-control and smartphone-use explain their academic performance. Computers in Human Behavior, 117, 106624. https://doi.org/10.1016/j.chb.2020.106624
Ulum, H. (2022). The effects of online education on academic success: A meta-analysis study. Education and Information Technologies, 27(1), 429–450. https://doi.org/10.1007/s10639-021-10740-8
Uvarov, A. Yu. (2019). The model of the digital school and the digital transformation of education. Researcher, (1–2), 22–37. (in Russ.).
Van Aert, R. C., Wicherts, J. M., & Van Assen, M. A. (2019). Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PloS one, 14(4), e0215052. https://doi.org/10.1371/journal.pone.0215052
Wang, J., Tigelaar, D. E. H., Zhou, T., & Admiraal, W. (2023). The effects of mobile technology usage on cognitive, affective, and behavioural learning outcomes in primary and secondary education: A systematic review with meta-analysis. Journal of Computer Assisted Learning, 39(2), 301–328. https://doi.org/10.1111/jcal.12759
Wang, M., Lwin, M. O., Cayabyab, Y. M. T. M., Hou, G., & You, Z. (2023). A Meta-Analysis of Factors Predicting Parental Mediation of Children’s Media Use Based on Studies Published Between 1992–2019. Journal of Child and Family Studies, 32(5), 1249–1260. https://doi.org/10.1007/s10826-022-02459-y
Xu, Y., & Fang, F. (2022). Internationalization of teacher education and the nation state: rethinking nationalization in Singapore: by Rita Z. Nazeer-Ikeda, London, Routledge, 2020, 262 pp., ISBN 978-0-367-22704-3. Asia Pacific Journal of Education, 0(0), 1–3. https://doi.org/10.1080/02188791.2022.2151740
Yang, J.-M., Sung, Y.-T., & Chang, K.-E. (2020). Use of Meta-Analysis to Uncover the Critical Issues of Mobile Inquiry-Based Learning. Journal of Educational Computing Research, 58(4), 715–746. https://doi.org/10.1177/0735633119879366
Yi, Y. J., You, S., & Bae, B. J. (2016). The influence of smartphones on academic performance: The development of the technology-to-performance chain model. Library Hi Tech, 34(3), 480–499. https://doi.org/10.1108/lht-04-2016-0038
Yu, D., Yan, Z., & He, X. (2022). Capturing knowledge trajectories of mobile learning research: A main path analysis. Education and Information Technologies, 27(5), 7257–7280. https://doi.org/10.1007/s10639-021-10869-6
Zhai, X., Zhang, M., Li, M., & Zhang, X. (2019). Understanding the relationship between levels of mobile technology use in high school physics classrooms and the learning outcome. British Journal of Educational Technology, 50(2), 750–766. https://doi.org/10.1111/bjet.12700
Zhao, K. (2020). Educating for wholeness, but beyond competences: Challenges to key-competences-based education in China. ECNU Review of Education, 3(3), 470–487. https://doi.org/10.1177/2096531120935463
Zheng, L., Li, X., Tian, L., & Cui, P. (2018). The effectiveness of integrating mobile devices with inquiry-based learning on students’ learning achievements: A meta-analysis. International Journal of Mobile Learning and Organisation, 12(1), 77–95. https://doi.org/10.1504/IJMLO.2018.089238
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2024 Yuliya L. Proekt, Elena B. Spasskaya, Nina O. Ivanushkina, Ol'ga S. Bocharova