Abstract

The Philippines has, for the recent several international assessments in mathematics, scored very low against the international standards. With the introduction of the K-to-12 curriculum, it is hoped that it would also create changes to the teaching and learning of mathematics anchored on its twin goals of critical thinking and problem solving. One great contributor to achieve this are the mathematics textbooks that are used in the classroom. The study aims to see the extent that the textbook items contribute to the desired outcome of exposing the student to Higher-Order Applications in the Real-World (HOARW) items, by assessing if the textbook items can be classified as HOARW. A framework for the assessment was created utilizing Problem Type (Non-application vs Application), Higher-Order Thinking Skills (With or without the use of technology), Contextual Feature (Fictional vs Realistic), and 21st Century Skill identification, as its layered lens. Textbook analysis was used as the research design. Items are categorized either as Example, Task, Exercise, or Test. The study yielded the following results: Examples and Exercises contributed to Lower Order Thinking items at 96.2 and 87.9 percent, respectively. Tasks mostly yielded Higher Order Thinking items at 67.3 percent. Overall, 98.7 percent of the items are Non-application problems, and 1.3 percent are considered application problems. Among the application problems, 33 percent are classified as Quasi-HOARW items, whereas 67 percent are HOARW items. The framework that was developed enables the proponent of the research to rewrite and improve the Quasi-HOARW items into HOARW items, by changes made to any of the layers of Higher-Order thinking or of Contextual Feature. The developed framework could help future textbook authors come up with better HOARW items and help expose the Filipino student to these types of items. Visual Text Analytics was used to conduct keyword extraction on the items yielding the most frequently used words of the authors, whether it invokes HOTS, and 21st Century skills. The framework was entered into a machine learning program to see if it could classify the items accordingly as intended.

References

Ahonen, A. K., & Kankaanranta, M. (2015). Assessment and Teaching of 21st Century Skills. In P. Griffin & E. Care (Eds.), Assessment and Teaching of 21st Century Skills. Springer Netherlands. https://doi.org/10.1007/978-94-017-9395-7
Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives (L. W. Anderson, D. R. Krathwohl, & B. S. (Benjamin S. Bloom (eds.); Complete ed.) [Book]. Longman.
Andrade, M. S. (2016). Curricular Elements for Learner Success—21st Century Skills. Journal of Education and Training Studies, 4(8), 143–149. https://doi.org/10.11114/jets.v4i8.1743
Aslan, S. (2015). Is learning by teaching effective in gaining 21st century skills? The views of pre-service science teachers. Educational Sciences: Theory & Practice, 15(6), 1441–1457. https://doi.org/10.12738/estp.2016.1.019
Basawapatna, A., Koh, K. H., Repenning, A., Webb, D. C., & Marshall, K. S. (2011). Recognizing computational thinking patterns. SIGCSE’11 - Proceedings of the 42nd ACM Technical Symposium on Computer Science Education, 245–250. https://doi.org/10.1145/1953163.1953241
BAYAZİT, İ. (2013). An Investigation of Problem Solving Approaches, Strategies, and Models Used by the 7th and 8th Grade Students when Solving Real-World Problems. Educational Sciences: Theory & Practice, 13(3), 1–25. https://doi.org/10.12738/estp.2013.3.1419
Bedir, H. (2019). Pre-service ELT Teachers’ Beliefs and Perceptions on 21st Century Learning and Innovation Skills (4Cs). Journal of Language and Linguistic Studies, 15(1), 231–246. https://doi.org/10.17263/jlls.547718
Beers, S. Z. (n.d.). 21st Century Skills: Preparing Students for Their Future. http://www.ncbi.nlm.nih.gov/pubmed/25950038
Boorman, D. (2015). Should Primary School Children be “Calculator Aware” or “Calculator Beware.” Teacher Education Advancement Network Journal, 7, 74–85.
Borba, R., & Selva, A. (2013). Analysis of the role of the calculator in Brazilian textbooks. ZDM - International Journal on Mathematics Education, 45(5), 737–750. https://doi.org/10.1007/s11858-013-0517-3
Bower, M., Wood, L. N., Lai, J. W. M., Howe, C., & Lister, R. (2017). Improving the computational thinking pedagogical capabilities of school teachers. Australian Journal of Teacher Education, 42(3), 53–72. https://doi.org/10.14221/ajte.2017v42n3.4
Care, E., Kim, H., Vista, A., & Anderson, K. (2018). Education system alignment for 21st century skills: Focus on assessment. Center for Universal Education at the Brookings Institution., 1–40. https://www.brookings.edu/wp-content/uploads/2018/11/Education-system-alignment-for-21st-century-skills-012819.pdf
Ceretkova, S., Sedivy, O., Molnar, J., & Petr, D. (2008). The Role and Assessment of Textbooks In Mathematics Education. Problems of Education in the 21st Century, 6(2), 27–38. https://doi.org/10.18546/HERJ.03.2.01
Chinedu, C. C., Kamin, Y., & O.S., O. (2015). Strategies for improving higher order thinking skills in teaching and learning of design and technology education. Journal of Technical Education and Training, 7(2), 35–43.
Chou, P. I., & Ting, H. J. (2016). How closely related are the national curriculum and the global dimension? A content analysis of the global dimension in elementary school textbooks in Taiwan. Asia Pacific Education Review, 17(3), 533–543. https://doi.org/10.1007/s12564-016-9437-6
DepEd Philippines. (2016). K to 12 Curriculum Guide: Mathematics (Grade 1 to Grade 10) (Issue July, p. 247). http://www.deped.gov.ph/sites/default/files/page/2017/English CG!.pdf
Drieger, P. (2013). Semantic Network Analysis as a Method for Visual Text Analytics. Procedia - Social and Behavioral Sciences, 79, 4–17. https://doi.org/10.1016/j.sbspro.2013.05.053
Fagginger Auer, M. F., Hickendorff, M., Van Putten, C. M., Béguin, A. A., & Heiser, W. J. (2016). Multilevel Latent Class Analysis for Large-Scale Educational Assessment Data: Exploring the Relation Between the Curriculum and Students’ Mathematical Strategies. Applied Measurement in Education, 29(2), 144–159. https://doi.org/10.1080/08957347.2016.1138959
Fan, L., Zhu, Y., & Miao, Z. (2013). Textbook research in mathematics education: Development status and directions. ZDM - International Journal on Mathematics Education, 45(5), 633–646. https://doi.org/10.1007/s11858-013-0539-x
Foster, J., & Yaoyuneyong, G. (2016). Teaching innovation: Equipping students to overcome real-world challenges. Higher Education Pedagogies, 1(1), 42–56. https://doi.org/10.1080/23752696.2015.1134195
Gainsburg, J. (2008). Real-world connections in secondary mathematics teaching. Journal of Mathematics Teacher Education, 11(3), 199–219. https://doi.org/10.1007/s10857-007-9070-8
Gatabi, A. R., Stacey, K., & Gooya, Z. (2012). Investigating grade nine textbook problems for characteristics related to mathematical literacy. Mathematics Education Research Journal, 24(4), 403–421. https://doi.org/10.1007/s13394-012-0052-5
Gazit, A., & Patkin, D. (2012). The way adults with orientation to mathematics teaching cope with the solution of everyday real-world problems. International Journal of Mathematical Education in Science and Technology, 43(2), 167–176. https://doi.org/10.1080/0020739X.2011.592614
Glasnovic Gracin, D. (2018). Requirements in mathematics textbooks: a five-dimensional analysis of textbook exercises and examples. International Journal of Mathematical Education in Science and Technology, 49(7), 1003–1024. https://doi.org/10.1080/0020739X.2018.1431849
Gretter, S., & Yadav, A. (2016). Computational Thinking and Media & Information Literacy: An Integrated Approach to Teaching Twenty-First Century Skills. TechTrends. https://doi.org/10.1007/s11528-016-0098-4
Griffin, P., McGaw, B., & Care, E. (2012). Assessment and Teaching of 21st Century Skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (Vol. 9789400723). Springer Netherlands. https://doi.org/10.1007/978-94-007-2324-5
Grover, S., & Pea, R. (2017). Computational thinking: A competency whose time has come. Computer Science Education: Perspectives on Teaching and Learning in School, December, 19–39. http://hub.mspnet.org/index.cfm/33300
Hadad, R., & Lawless, K. A. (2014). Assessing Computational Thinking. In Encyclopedia of Information Science and Technology, Third Edition. https://doi.org/10.4018/978-1-4666-5888-2.ch150
Hammill, L. (2010). The Interplay of Text , Symbols , and Graphics in Mathematics Education. Transformative Dialogues: Teaching & Learning Journal, 3(3), 1–8.
Hart Research Associates. (2010). Raising the Bar: Employers Views on College Learning in the Wake of the Economic Downturn. http://www.aacu.org/leap/documents/2009_EmployerSurvey.pdf
Heong, Y. M., Othman, W. B., Yunos, J. B. M., Kiong, T. T., Hassan, R. Bin, & Mohamad, M. M. B. (2011). The Level of Marzano Higher Order Thinking Skillsamong Technical Education Students. International Journal of Social Science and Humanity, 1(2), 121–125. https://doi.org/10.7763/IJSSH.2011.V1.20
Heong, Y. M., Yunos, J. M., Othman, W., Hassan, R., Kiong, T. T., & Mohamad, M. M. (2012). The Needs Analysis of Learning Higher Order Thinking Skills for Generating Ideas. Procedia - Social and Behavioral Sciences, 59, 197–203. https://doi.org/10.1016/j.sbspro.2012.09.265
Iqbal, J., & Khan, W. (2013). A Review of English Textbook at Secondary Level in the Province of Khyber Pakhtunkhwa, Pakistan.
Iter, N. (2017). Using Performance Task-GRASPS to Assess Student Performance in Higher Education Courses. American Journal of Educational Research, 5(5), 552–558. https://doi.org/10.12691/education-5-5-12
Jänicke, S., Franzini, G., Cheema, M. F., & Scheuermann, G. (2017). Visual Text Analysis in Digital Humanities. Computer Graphics Forum, 36(6), 226–250. https://doi.org/10.1111/cgf.12873
KAYHAN ALTAY, M., KURT ERHAN, G., & BATI, E. (2020). Contexts Used for Real Life Connections in Mathematics Textbook for 6th Graders. İlköğretim Online, 19(1), 310–323. https://doi.org/10.17051/ilkonline.2020.656880
Kim, D.-J., Bae, S.-C., Kim, W., Lee, D.-H., & Choi, S.-H. (2015). Analysis of mathematical tasks provided by storytelling mathematics textbooks. Communications of Mathematical Education, 29(3), 281–300. https://doi.org/10.7468/jksmee.2015.29.3.281
Kim, S., Raza, M., & Seidman, E. (2019). Improving 21st-century teaching skills: The key to effective 21st-century learners. Research in Comparative and International Education, 14(1), 99–117. https://doi.org/10.1177/1745499919829214
Kivunja, C. (2014). Teaching Students to Learn and to Work Well with 21st Century Skills: Unpacking the Career and Life Skills Domain of the New Learning Paradigm. International Journal of Higher Education, 4(1), 1–11. https://doi.org/10.5430/ijhe.v4n1p1
Kivunja, C. (2015). Exploring the Pedagogical Meaning and Implications of the 4Cs “Super Skills” for the 21st Century through Bruner’s 5E Lenses of Knowledge Construction to Improve Pedagogies of the New Learning Paradigm. Creative Education, 06(02), 224–239. https://doi.org/10.4236/ce.2015.62021
Kong, S. C., Chan, T. W., Griffin, P., Hoppe, U., Huang, R., Kinshuk, Looi, C. K., Milrad, M., Norris, C., Nussbaum, M., Sharples, M., So, W. M. W., Soloway, E., & Yu, S. (2013). E-learning in school education in the coming 10 years for developing 21st century skills: Critical research issues and policy implications. Educational Technology and Society.
Kristanto, Y. D., & Santoso, E. B. (2020). Towards a mathematics textbook for supporting 21st century learning: The student perspective. Journal of Physics: Conference Series, 1657(1), 012037. https://doi.org/10.1088/1742-6596/1657/1/012037
Leonard, J., Mitchell, M., Barnes-Johnson, J., Unertl, A., Outka-Hill, J., Robinson, R., & Hester-Croff, C. (2018). Preparing Teachers to Engage Rural Students in Computational Thinking Through Robotics, Game Design, and Culturally Responsive Teaching. Journal of Teacher Education, 69(4), 386–407. https://doi.org/10.1177/0022487117732317
Levin-Goldberg, J. (2012). Teaching Generation TechX with the 4Cs: Using Technology to Integrate 21st Century Skills. Journal of Instructional Research, 1, 59–66. https://doi.org/10.9743/JIR.2013.8
Lew, H.-C., & Jeong, S.-Y. (2014). Key factors for Successful Integration of Technology into the Classroom: Te...: EBSCOhost. Electronic Journal of Mathematics and Technology, 19. http://ezproxy.georgetowncollege.edu:2058/ehost/detail/detail?vid=14&sid=1b51f4b7-8ae8-4cb4-b2ec-170d9f2a7588%40sessionmgr107&hid=119&bdata=#AN=99721169&db=a9h
Lisa O’Keeffe, & O’Donoghue, J. (2010). The importance of Mathematics Textbook Analysis in relation to students learning and understanding of mathematic. January.
Liu, S., Zhou, M. X., Pan, S., Qian, W., Cai, W., & Lian, X. (2009). Interactive, topic-based visual text summarization and analysis. Proceeding of the 18th ACM Conference on Information and Knowledge Management - CIKM ’09, 543. https://doi.org/10.1145/1645953.1646023
Livingstone, S. (2012). Critical reflections on the benefits of ICT in education. Oxford Review of Education, 38(1), 9–24. https://doi.org/10.1080/03054985.2011.577938
Lopes, A. J., Vanegas, Y., & Giménez, J. (2018). Redesigning open tasks in mathematics textbook. Proceedings of the Second International Conference on Mathematics Textbook Research and Development, December, 318–325.
Mainali, B. P. (2013). Higher Order Thinking in Education. Academic Voices: A Multidisciplinary Journal, 2(1), 5–10. https://doi.org/10.3126/av.v2i1.8277
Manopo, M., & Lisarani, V. (2021). Comparative Analysis of Non-Routine Problems in Mathematics Textbooks of Indonesia and Singapore. Edumatika : Jurnal Riset Pendidikan Matematika, 4(1), 158–165. https://doi.org/10.32939/ejrpm.v4i1.750
Margana, M., & Widyantoro, A. (2017). Developing English Textbooks Oriented to Higher Order Thinking Skills for Students of Vocational High Schools in Yogyakarta. Journal of Language Teaching and Research, 8(1), 26. https://doi.org/10.17507/jltr.0801.04
Marshall, J. C., & Horton, R. M. (2011). The Relationship of Teacher-Facilitated, Inquiry-Based Instruction to Student Higher-Order Thinking. School Science and Mathematics, 111(3), 93–101. https://doi.org/10.1111/j.1949-8594.2010.00066.x
McLoughlin, C., & Lee, M. J. W. (2010). Personalised and self regulated learning in the Web 2.0 era: International exemplars of innovative pedagogy using social software. Australasian Journal of Educational Technology, 26(1), 28–43. https://doi.org/10.14742/ajet.1100
Miller, J., & Benderlioglu, Z. (2020). New Tools for Quantifying Textbook Design for Biology Education (Issue February).
Mullis, I. V. S., & Martin, M. O. (2019). TIMSS 2015 Completed TIMSS 2019 Underway. October 2017.
Nagnedrarao, B., & Selangor, S. A. (2017). Mathematics Teachers ’ Perception Using Textbook To Promote. Science International (Labore), 29(4), 965–970.
Okeeffe, L. (2013). A Framework for Textbook Analysis. International Review of Contemporary Learning Research, 2(1), 1–13. https://doi.org/10.12785/irclr/020101
Ottenbreit-Leftwich, A. T., Glazewski, K. D., Newby, T. J., & Ertmer, P. A. (2010). Teacher value beliefs associated with using technology: Addressing professional and student needs. Computers and Education, 55(3), 1321–1335. https://doi.org/10.1016/j.compedu.2010.06.002
OUCEA Oxford. (2019). Official launch of the Programme for International Student Assessment 2021 Mathematics Framework. https://www.youtube.com/watch?v=b68dRraybAU&t=3461s
Pedro, J., & Marques, S. (2007). Proportion in school mathematics textbooks : A comparative study. Most, 5, 1–22.
Petersson, J., Sayers, J., Rosenqvist, E., & Andrews, P. (2021). Two novel approaches to the content analysis of school mathematics textbooks. International Journal of Research & Method in Education, 44(2), 208–222. https://doi.org/10.1080/1743727X.2020.1766437
Polikoff, M. S., Campbell, S. E., Rabovsky, S., Koedel, C., Le, Q. T., Hardaway, T., & Gasparian, H. (2020). The formalized processes districts use to evaluate mathematics textbooks. Journal of Curriculum Studies, 52(4), 451–477. https://doi.org/10.1080/00220272.2020.1747116
Pratama, G. S., & Retnawati, H. (2018). Urgency of Higher Order Thinking Skills (HOTS) Content Analysis in Mathematics Textbook. Journal of Physics: Conference Series, 1097(1). https://doi.org/10.1088/1742-6596/1097/1/012147
Rezat, S. (2010). the Utilization of Mathematics Textbooks As Instruments for Learning. Procedings of CERME, October, 1260–1269.
Rice, E. (2011). Reframing Student Outcomes to Develop 21st Century Skills. Knowledge Brief, 4. http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED533012
Rios, J. A., Ling, G., Pugh, R., Becker, D., & Bacall, A. (2020). Identifying Critical 21st-Century Skills for Workplace Success: A Content Analysis of Job Advertisements. Educational Researcher, 49(2), 80–89. https://doi.org/10.3102/0013189X19890600
Rohrer, D., Dedrick, R. F., & Hartwig, M. K. (2020). The Scarcity of Interleaved Practice in Mathematics Textbooks. Educational Psychology Review, 32(3), 873–883. https://doi.org/10.1007/s10648-020-09516-2
Román-González, M., Pérez-González, J. C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2016.08.047
Ruiz-Primo, M. (2009). Towards a framework for assessing 21st century science skills. The National Academies. http://sites.nationalacademies.org/dbasse/cs/groups/dbassesite/documents/webpage/dbasse_072612.pdf
Saido, G. M., Siraj, S., Nordin, A. B. Bin, & Al_Amedy, O. S. (2015). Higher Order Thinking Skills among Secondary School Students in Science Learning. Malaysian Online Journal of Educational Sciences, 3(3), 13–20.
Salha, S., Barakat, A., & Nassar, D. (2020). The availability of Problem Solving, Communication, Mathematical Representation Standards in the Tenth Grade Mathematics Textvook in Palestine. International Academic Journal in Educational and Psychological Sciences, 1(May), 208–254.
Schmidt, W. H. (2012). Measuring Content Through Textbooks: The Cumulative Effect of Middle-School Tracking. In G. Gueudet, B. Pepin, & L. Trouche (Eds.), From Text to “Lived” Resources: Mathematics Curriculum Materials and Teacher Development. Springer Netherlands. https://doi.org/10.1007/978-94-007-1966-8
Şener, N., Türk, C., & Taş, E. (2015). Improving Science Attitude and Creative Thinking through Science Education Project: A Design, Implementation and Assessment. Journal of Education and Training Studies, 3(4), 57–67. https://doi.org/10.11114/jets.v3i4.771
Sherman, M. F., Cayton, C., Walkington, C., & Funsch, A. (2020). An Analysis of Secondary Mathematics Textbooks With Regard to Technology Integration. Journal for Research in Mathematics Education, 51(3), 361–374. https://doi.org/10.5951/jresemtheduc-2020-0005
Shield, M., & Dole, S. (2013). Assessing the potential of mathematics textbooks to promote deep learning. Educational Studies in Mathematics, 82(2), 183–199. https://doi.org/10.1007/s10649-012-9415-9
Shields, R., & Chugh, R. (2018). Preparing Australian High School Learners with 21st Century Skills. 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), December, 1101–1106. https://doi.org/10.1109/TALE.2018.8615207
Siddiq, F., Gochyyev, P., & Wilson, M. (2017). Learning in Digital Networks – ICT literacy: A novel assessment of students’ 21st century skills. Computers & Education, 109, 11–37. https://doi.org/10.1016/j.compedu.2017.01.014
Sievert, H., van den Ham, A.-K., Niedermeyer, I., & Heinze, A. (2019). Effects of mathematics textbooks on the development of primary school children’s adaptive expertise in arithmetic. Learning and Individual Differences, 74(February), 101716. https://doi.org/10.1016/j.lindif.2019.02.006
Silva, E. (2008). Measuring skills for 21st-century learning. Phi Delta Kappan. https://doi.org/10.1177/003172170909000905
Smit, L. S. (2016). A better understanding of 21st century skills in mathematics education and a view on these skills in current practice. Mathematics Enthusiast, 19(8), 424–425. https://doi.org/10.1016/j.mattod.2016.08.009
Smith, C., & Morgan, C. (2016). Curricular orientations to real‐world contexts in mathematics. The Curriculum Journal, 27(1), 24–45. https://doi.org/10.1080/09585176.2016.1139498
Stillman, G. A., & Galbraith, P. L. (1998). Applying mathematics with real world connections: Metacognitive characteristics of secondary students. Educational Studies in Mathematics, 36(2), 157–194. https://doi.org/10.1023/A:1003246329257
Tan, K. J., Ismail, Z., & Abidin, M. (2018). A comparative analysis on cognitive domain for the Malaysian primary four textbook series. Eurasia Journal of Mathematics, Science and Technology Education, 14(4), 1273–1286. https://doi.org/10.29333/ejmste/82625
Tan, S. Y., & Halili, S. H. (2015). Effective Teaching of Higher-Order Thinking (HOT) in Education. The Online Journal of Distance Education and E-Learning, 3(2), 41–47.
Tanujaya, B., & Mumu, J. (2021). Reconstruction of HOTS problems based on questions in mathematics textbook. Journal of Physics: Conference Series, 1832(1), 012052. https://doi.org/10.1088/1742-6596/1832/1/012052
Teo, P. (2019). Teaching for the 21st century: A case for dialogic pedagogy. In Learning, Culture and Social Interaction (Vol. 21, pp. 170–178). Elsevier Ltd. https://doi.org/10.1016/j.lcsi.2019.03.009
Thang, S. M., Sim, L. Y., Mahmud, N., Lin, L. K., Zabidi, N. A., & Ismail, K. (2014). Enhancing 21st Century Learning Skills Via Digital Storytelling: Voices of Malaysian Teachers and Undergraduates. Procedia - Social and Behavioral Sciences, 118, 489–494. https://doi.org/10.1016/j.sbspro.2014.02.067
Tondeur, J., Van Braak, J., Sang, G., Voogt, J., Fisser, P., & Ottenbreit-Leftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence. Computers and Education, 59(1), 134–144. https://doi.org/10.1016/j.compedu.2011.10.009
Tuan Soh, T. M., Osman, K., & Arsad, N. M. (2012). M-21CSI: A Validated 21st Century Skills Instrument for Secondary Science Students. Asian Social Science, 8(16), 38–44. https://doi.org/10.5539/ass.v8n16p38
Ulusoy, F., & Incikabi, L. (2020). Middle School Teachers ’ Use of Compulsory Textbooks in Instruction of Mathematics. International Journal for Mathematics Teaching and Learning, 21(1), 1–18.
Urbani, J., Roshandel, S., Michaels, R., & Truesdell, E. (2017). Developing and Modeling 21st-Century Skills with Preservice Teachers. Teacher Education Quarterly, 44(4), 27–50.
Usiskin, Z. (2013). Studying textbooks in an information age—a United States perspective. ZDM, 45(5), 713–723. https://doi.org/10.1007/s11858-013-0514-6
Valverde, G. A., Bianchi, L. J., Wolfe, R. G., Schmidt, W. H., & Houang, R. T. (2002). According to the Book. In According to the Book. https://doi.org/10.1007/978-94-007-0844-0
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577–588. https://doi.org/10.1016/j.chb.2017.03.010
Vijayaratnam, P. (2012). Developing Higher Order Thinking Skills and Team Commitment via Group Problem Solving: A Bridge to the Real World. Procedia - Social and Behavioral Sciences. https://doi.org/10.1016/j.sbspro.2012.11.247
Vincent, J., & Stacey, K. (2008). Do mathematics textbooks cultivate shallow teaching? Applying the TIMSS Video Study criteria to Australian eighth-grade mathematics textbooks. Mathematics Education Research Journal, 20(1), 82–107. https://doi.org/10.1007/BF03217470
Vreeburg Izzo, M., Yurick, A., Nagaraja, H. N., & Novak, J. A. (2010). Effects of a 21st-Century Curriculum on Students’ Information Technology and Transition Skills. Career Development for Exceptional Individuals, 33(2), 95–105. https://doi.org/10.1177/0885728810369348
Weinberg, A., Wiesner, E., Benesh, B., & Boester, T. (2012). Undergraduate Students’ Self-Reported Use of Mathematics Textbooks. PRIMUS, 22(2), 152–175. https://doi.org/10.1080/10511970.2010.509336
Wilcox, D., Liu, J. C., Thall, J., & Howley, T. (2017). Integration of Teaching Practice for Students’ 21st Century Skills: Faculty Practice and Perception. International Journal of Technology in Teaching and Learning, 13(2), 55–77.
Wood, L. N. (2010). Graduate capabilities: putting mathematics into context. International Journal of Mathematical Education in Science and Technology, 41(2), 189–198. https://doi.org/10.1080/00207390903388607
Yang, D. C., Tseng, Y. K., & Wang, T. L. (2017). A comparison of geometry problems in middle-grade mathematics textbooks from Taiwan, Singapore, Finland, and the United States. Eurasia Journal of Mathematics, Science and Technology Education, 13(7), 2841–2857. https://doi.org/10.12973/eurasia.2017.00721a
Yu, W. C. W., Lin, C. C., Ho, M. H., & Wang, J. (2015). Technology facilitated PBL pedagogy and its impact on nursing student’s academic achievement and critical thinking dispositions. Turkish Online Journal of Educational Technology, 14(1), 97–107.
Zhu, Y., & Fan, L. (2006). Focus on the Representation of Problem Types in Intended Curriculum: A Comparison of Selected Mathematics Textbooks from Mainland China and the United States. International Journal of Science and Mathematics Education, 4(4), 609–626. https://doi.org/10.1007/s10763-006-9036-9