Efficacy of an Online Reading Intervention Program “Literacy Pro” in Classifying Students as “At Risk” and “Not at Risk” among ESL School Students—A Logistic Regression Analysis
The purpose of this study is to test the
classification accuracy of an online reading comprehension program “Literacy
Pro” in classifying students as “at risk” and “not at risk”. Data was collected
from the school records of an international school operating in Dubai, UAE. The
students of this school are non-native English speakers or they learn English
as a Second Language (ESL). The sample was taken from two grades—grade 7 and
grade 10. A logistic regression analysis (simple and multiple) was done to test
the classification accuracy in terms of the sensitivity and specificity rule.
The results of the current study show a sensitivity level of 0.76 and 0.94 for grade
7 and 10 respectively, whereas the corresponding specificity levels are 0.99
and 0.97 respectively. The overall accuracy of the model for grade 7 and 10 is
89.9% and 94.4% respectively in simple logistic regression whereas the
corresponding values are 89.9% and 95.6% in multiple logistic regressions. Therefore, it can be
concluded that the reading intervention measure of “Literacy Pro” is highly
accurate in classifying students as “at risk” and “not at risk”. The area under
the curve (AUC) value was obtained as 0.8754 and 0.9512 for grade 7 and 10
respectively for the ROC curves of the model. Hence, the classification
accuracy of grade 10 is excellent and that of grade 7 is good as per the
generally accepted standards for the online reading comprehension program
“Literacy Pro”.
References
[1]
Al-Shehri, S., & Gitsaki, C. (2010). Online Reading: A Preliminary Study of the Impact of Integrated and Split-Attention Formats on l2 Students’ Cognitive Load. ReCALL, 22, 356-375. https://doi.org/10.1017/S0958344010000212
[2]
Anderson, R. C., Wilson, P. T., & Fielding, L. G. (1988). Growth in Reading and How Children Spend Their Time Outside of School. Reading Research Quarterly, 23, 285-303. http://www.jstor.org/stable/748043 https://doi.org/10.1598/RRQ.23.3.2
[3]
Biancarosa, G., & Snow, C. E. (2004). Reading Next: A Vision for Action and Research in Middle and High School Literacy (2nd ed., p. 49). The Alliance for Excellent Education.
[4]
Bishop, M. J., & Santoro, L. E. (2006). Evaluating Beginning Reading Software for At-Risk Learners. Psychology in the Schools, 43, 57-70. https://doi.org/10.1002/pits.20129
[5]
Calhoon, M. B. (2005). Effects of a Peer-Mediated Phonological Skill and Reading Comprehension Program on Reading Skill Acquisition for Middle School Students with Reading Disabilities. Journal of Learning Disabilities, 38, 424-433. https://doi.org/10.1177/00222194050380050501
[6]
Carney, R. (2010). Using Web-Based Instruction to Teach Music Theory in the Piano Studio: Defining, Designing, and Implementing an Integrative Approach. https://www.researchgate.net/publication/234720493_Using_Web-Based_Instruction_to_Teach_Music_Theory_in_the_Piano_Studio_Defining_Designing_and_Implementing_an_Integrative_Approach
[7]
Catts, H. W., Petscher, Y., Schatschneider, C., Sittner Bridges, M., & Mendoza, K. (2009). Floor Effects Associated with Universal Screening and Their Impact on the Early Identification of Reading Disabilities. Journal of Learning Disabilities, 42, 163-176. https://doi.org/10.1177/0022219408326219
[8]
Chun-Min, W., & Thomas, C. R. (2007). The Meaning of Culture in Online Education: Implications for Teaching, Learning and Design. In E. Andrea (Ed.), Globalized E-Learning Cultural Challenges (pp. 1-17). IGI Global. https://doi.org/10.4018/978-1-59904-301-2.ch001
[9]
Coiro, J. (2003). Reading Comprehension on the Internet: Expanding Our Understanding of Reading Comprehension to Encompass New Literacies. Reading Teacher, 56, 458-464.
[10]
Cole, J. M., & Hilliard, V. R. (2006). The Effects of Web-Based Reading Curriculum on Children’s Reading Performance and Motivation. Journal of Educational Computing Research, 34, 353-380. https://doi.org/10.2190/H43W-1N3U-027J-07V5
[11]
Compton, D., Fuchs, D., Fuchs, L., & Bryant, J. (2006). Selecting At-Risk Readers in First Grade for Early Intervention: A Two-Year Longitudinal Study of Decision Rules and Procedures. Journal of Educational Psychology, 98, 394-409. https://doi.org/10.1037/0022-0663.98.2.394
[12]
David, J. L. (2010). What Research Says about Closing the Vocabulary Gap. Educational Leadership, 67, 85-86.
[13]
Eilon, B., & Kliachko, S. (2004). Perceptions of the Teacher’s Roles by Prospective Elementary School Science Teachers in a Web-Based Biology Course. Journal of Technology and Teacher Education, 12, 339-360. https://eric.ed.gov/?id=EJ723698
[14]
Englert, C. S., Manalo, M., & Zhao, Y. (2004). I Can Do It Better on the Computer: The Effects of Technology-Enabled Scaffolding on Young Writers’ Composition. Journal of Special Education Technology, 19, 5-21. https://doi.org/10.1177/016264340401900101
[15]
Englert, C. S., Zhao, Y., Collings, N., & Romig, N. (2005). Learning to Read Words: The Effects of Internet-Based Software on the Improvement of Reading Performance. Remedial and Special Education, 26, 357-371. https://doi.org/10.1177/07419325050260060601
[16]
Eunice Kennedy Shriver National Institute of Child Health and Human Development, N., DHHS (2001). Put Reading First: The Research Building Blocks for Teaching Children to Read (N/A). https://www.nichd.nih.gov/publications/product/239
[17]
Fasting, R., & Lyster, S.-A. (2005). The Effects of Computer Technology in Assisting the Development of Literacy in Young Struggling Readers and Spellers. European Journal of Special Needs Education, 20, 21-40. https://doi.org/10.1080/0885625042000319061
[18]
Geske, A., & Ozola, A. (2009). Different Influence of Contextual Educational Factors on Boys’ and Girls’ Reading Achievement. US-China Education Review, 6, 38-44. https://files.eric.ed.gov/fulltext/ED505734.pdf
[19]
Glover, T., & Albers, C. (2007). Considerations for Evaluating Universal Screening Assessments. Journal of School Psychology, 45, 117-135. https://doi.org/10.1016/j.jsp.2006.05.005
[20]
Goodfellow, R., & Lea, M. R. (2005). Supporting Writing for Assessment in Online Learning. Assessment & Evaluation in Higher Education, 30, 261-271. https://doi.org/10.1080/02602930500063835
[21]
Harmon, J., Wood, K., Hedrick, W., Vintinner, J., & Willeford, T. (2009). Interactive Word Walls: More than Just Reading the Writing on the Walls. Journal of Adolescent & Adult Literacy, 52, 398-408. https://doi.org/10.1598/JAAL.52.5.4
[22]
Harrison, C. (2009). On-Line Learning & Thinking in Science: Uncovering How Secondary School Students Learn about Velocity in a Web-Based Environment. Doctor of Education, Mills College. https://www.researchgate.net/publication/241516183_On-line_learning_thinking_in_science_Uncovering_how_secondary_school_students_learn_about_velocity_in_a_Web-based_environment
[23]
Herold, B. (2014). Digital Reading Poses Learning Challenges for Students. Education Week, 33, 24-25.
[24]
Inceçay, G. (2013). Metacognitive Online Reading Strategies Applied by EFL Students. Journal of Theory and Practice in Education, 9, 390-407. https://dergipark.org.tr/en/download/article-file/63367
[25]
Jenkins, J., Hudson, R., & Johnson, E. (2007). Screening for At-Risk Readers in a Response to Intervention Framework. School Psychology Review, 36, 582-599. https://www.academia.edu/27687650/Screening_for_At_Risk_Readers_in_a_Response_to_Intervention_Framework https://doi.org/10.1080/02796015.2007.12087919
[26]
Johnson, E., Mellard, D., Fuchs, D., & McKnight, M. (2006). Responsiveness to Intervention (RTI): How to Do It. [RTI Manual]. https://eric.ed.gov/?id=ED496979
[27]
Jones, C. (2001). The Relationship between Writing Centers and Improvement in Writing Ability: An Assessment of the Literature. Education, 122, 3-13.
[28]
Kent, S. C., Wanzek, J., & Yun, J. (2019). Screening in the Upper Elementary Grades: Identifying Fourth-Grade Students At-Risk for Failing the State Reading Assessment. Assessment for Effective Intervention, 44, 160-172. https://doi.org/10.1177/1534508418758371
[29]
Kern, R. G. (1989). Second Language Reading Strategy Instruction: Its Effects on Comprehension and Word Inference Ability. The Modern Language Journal, 73, 135-149. https://doi.org/10.1111/j.1540-4781.1989.tb02535.x
[30]
Kilgus, S. P., Methe, S. A., Maggin, D. M., & Tomasula, J. L. (2014). Curriculum-Based Measurement of Oral Reading (R-CBM): A Diagnostic Test Accuracy Meta-Analysis of Evidence Supporting Use in Universal Screening. Journal of School Psychology, 52, 377-405. https://doi.org/10.1016/j.jsp.2014.06.002
[31]
Lamb, A., & Johnson, L. (2010). Interactives: Dynamic Learning Environments. School Library Monthly, 26, 41-44. http://hdl.handle.net/1805/8652
[32]
Lea, M., & Jones, S. (2011). Digital Literacies in Higher Education: Exploring Textual and Technological Practice. Studies in Higher Education, 36, 377-393. https://doi.org/10.1080/03075071003664021
[33]
Littleton, K., Wood, C., & Chera, P. (2006). Interactions with Talking Books: Phonological Awareness Affects Boys’ Use of Talking Books. Journal of Computer Assisted Learning, 22, 382-390. https://doi.org/10.1111/j.1365-2729.2006.00183.x
[34]
Macaruso, P., & Walker, A. (2008). The Efficacy of Computer-Assisted Instruction for Advancing Literacy Skills in Kindergarten Children. Reading Psychology, 29, 266-287. https://doi.org/10.1080/02702710801982019
[35]
Massey, S. (2014). Making the Case for Using Informational Text in Preschool Classrooms. Creative Education, 5, 396-401. https://doi.org/10.4236/ce.2014.56049
[36]
Mullis, I. V. S., Kennedy, A. M., Martin, M. O., & Sainsbury, M. (2004). PIRLS 2006 Assessment Framework and Specifications: Progress in International Reading Literacy Study. https://eric.ed.gov/?id=ED484227
[37]
Palmer, D. (2006). A Motivational View of Constructivist-Informed Teaching. International Journal of Science Education, 27, 1853-1881. https://doi.org/10.1080/09500690500339654
[38]
Park, H.-R., & Kim, D. (2011). Reading Strategy Use by English as a Second Language Learners in Online Reading Tasks. Computers & Education, 57, 2156-2166. https://doi.org/10.1016/j.compedu.2011.05.014
[39]
Perlman, C., Weston, C., & Gisel, E. (2010). Enabling Meaningful Learning through Web-Based Instruction with Occupational Therapy Students. Educational Technology Research and Development, 58, 191-210. https://doi.org/10.1007/s11423-008-9097-2
[40]
Reutzel, D. R., & Cooter, R. B. (2004). Teaching Children to Read: Putting the Pieces Together (4th ed.). Merrill/Prentice-Hall Publishing Company.
[41]
Sadik, A. (2008). Digital Storytelling: A Meaningful Technology-Integrated Approach for Engaged Student Learning. Educational Technology Research and Development, 56, 487-506. https://doi.org/10.1007/s11423-008-9091-8
[42]
Schmar-Dobler, E. (2003). Reading on the Internet: The Link between Literacy and Technology. Journal of Adolescent and Adult Literacy, 47, 80-85. https://www.researchgate.net/publication/287009870_Reading_on_the_Internet_The_Link_between_Literacy_and_Technology
[43]
Sherman, D., Kleiman, G., & Peterson, K. (2004). Technology and Teaching Children to Read. https://eric.ed.gov/?id=ED485613
[44]
Smith, G. E., & Throne, S. (2007). Differentiating Instruction with Technology in K-5 Classrooms. International Society for Technology in Education.
[45]
Sternberg, B. J., Kaplan, K. A., & Borck, J. E. (2007). Enhancing Adolescent Literacy Achievement through Integration of Technology in the Classroom. Reading Research Quarterly, 42, 416-420. http://www.jstor.org/stable/20068306 https://doi.org/10.1598/RRQ.42.3.6
[46]
Taylor, T., & Ward, I. (1998). Literacy Theory in the Age of Internet. Columbia University Press.
[47]
Topping, K. (1999). Formative Assessment of Reading Comprehension by Computer: Advantages and Disadvantages of the Accelerated Reader Software. http://www.readingonline.org/critical/topping
[48]
White, S. A. (2004). Reading Strategies Training in a Japanese University English as a Foreign Language Setting. Doctor of Philosophy, University of Kansas, ProQuest Dissertations Publishing. https://search.proquest.com/docview/305170514?pq-origsite=gscholar&fromopenview=true
[49]
Yagci, T. (2014). Mobile Social Media Challenges Digital Natives in EFL Learning. Journal of Educational & Instructional Studies in the World, 4, 49-53. http://oaji.net/journal-detail.html?number=1759
[50]
Zhou, X. H., Obuchowski, N., & McClish, D. (2002). Statistical Methods in Diagnostic Medicine (2nd ed.). Wiley. https://doi.org/10.1002/9780470317082
