Missing data are always an issue in community-based longitudinal studies, calling into question the representativeness of samples and bias in conclusions, the research has generated. This may be due to the difficulty of implementing random sampling procedures in these studies and/or the inherent difficulty in sampling hard-to-reach segments of the population being studied. In fact, the ability to accurately study hard-to-reach populations in light of potential bias created by missing data remains an open question. In this study, missing data are defined as both failure to interview potential research participants identified in the sampling frame and failure to retain enrolled research participants longitudinally. Using the sample from the Mobile Youth Survey, a multiple-cohort, longitudinal study of adolescents living in highly impoverished neighborhoods in Mobile, Alabama, we examined sample representativeness and dropout to determine whether missing data led to a nonrepresentative, and therefore, biased sample. Results indicate that even though random procedures are not strictly used to draw the sample, (a) the sample appears to be largely representative of the population that was studied, and (b) attrition is largely uncorrelated with characteristics of those who dropped out. This suggests that it is possible to study with validity hard-to reach populations in community settings.
References
[1]
Gorin, S., Hooper, C.A., Dyson, C. and Cabral, C. (2008) Ethical Challenges in Conducting Research with Hard to Reach Families. Child Abuse Review, 17, 275-287.
https://doi.org/10.1002/car.1031
[2]
Shaghaghi, A., Bhopal, R.S. and Sheikh, A. (2011) Approaches to Recruiting “Hard-to-Reach” Populations into Research: A Review of the Literature. Health Promotion, 1, 1-9.
[3]
Freimuth, V.S. and Mettger, W. (1990) Is There a Hard-to-Reach Audience? Public Health Reports, 105, 232-238.
[4]
Lambert, E. and Wiebel, W. (1990) Introduction. In: Lambert, E. and Wiebel, W., Eds., The Collection and Interpretation of Data from Hidden Populations, National Institute on Drug Abuse Research Monograph 98, Government Printing Office, Washington DC, 1-3.
[5]
Haley, D.F., Lucas, J., Golin, C.E., Wang, J., Hughes, J.P., Emel, L., et al. (2014) Retention Strategies and Factors Associated with Missed Visits among Low Income Women at Increased Risk of HIV Acquisition in the US (HPTN 064). AIDS Patient Care and STDs, 28, 206-217. https://doi.org/10.1089/apc.2013.0366
[6]
Katz, K.S., El-Mohandes, A., Johnson, D.M., Jarrett, M., Rose, A. and Cober, M. (2001) Retention of Low Income Mothers in a Parenting Intervention Study. Journal of Community Health, 26, 203-218. https://doi.org/10.1023/A:1010373113060
[7]
Sullivan, C.M., Rumptz, M.H., Campbell, R., Eby, K.K. and Davidson, W.S. (1996) Retaining Participants in Longitudinal Community Research: A Comprehensive Protocol. The Journal of Applied Behavioral Science, 32, 262-276.
https://doi.org/10.1177/0021886396323002
[8]
Reiss, F. (2013) Socioeconomic Inequalities and Mental Health Problems in Children and Adolescents: A Systematic Review. Social Science & Medicine, 90, 24-31.
[9]
Nandi, A., Glymour, M.M. and Subramanian, S.V. (2014) Associations among Socioeconomic Status, Health Behaviors, and All-Cause Mortality in the United States. Epidemiology, 25, 170-177. https://doi.org/10.1097/EDE.0000000000000038
[10]
Entwisle, D.R., Alexander, K.L. and Olson, L.S. (1994) The Gender Gap in Math: Its Possible Origins in Neighborhood Effects. American Sociological Review, 59, 822-838. https://doi.org/10.2307/2096370
[11]
Alspaugh, J.W. (1991) Out-of-School Environmental Factors and Elementary Achievement in Mathematics and Reading. Journal of Research and Development in Education, 24, 53-55.
[12]
Balli, S.J., Demo, D.H. and Wedman, J.F. (1998) Family Involvement with Children’s Homework: An Intervention in the Middle Grades. Family Relations, 47, 149-157. https://doi.org/10.2307/585619
[13]
Graham, J.W., Hofer, S.M., Donaldson, S.I., MacKinnon, D.P. and Schafer, J.L. (1997) Analysis with Missing Data in Prevention Research. In: Bryant, K.J., Windle, M. and West, S.G., Eds., The Science of Prevention: Methodological Advances from Alcohol and Substance Abuse Research, American Psychological Association, Washington DC. https://doi.org/10.1037/10222-010
[14]
Do, D.P., Frank, R. and Finch, B.K. (2012) Does SES Explain More of the Black/ White Health Gap than We Thought? Revisiting Our Approach toward Understanding Racial Disparities in Health. Social Science & Medicine, 74, 1385-1393.
[15]
Braveman, P.A., Cubbin, C., Egerter, S., Chideya, S., Marchi, K.S., Metzler, M. and Posner, S. (2005) Socioeconomic Status in Health Research: One Size Does Not Fit All. JAMA, 294, 2879-2888. https://doi.org/10.1001/jama.294.22.2879
[16]
Hatchett, B.F., Holmes, K., Duran, D.A. and Davis, C. (2000) African Americans and Research Participation: The Recruitment Process. Journal of Black Studies, 30, 664-675. https://doi.org/10.1177/002193470003000502
[17]
Kerkorian, D., Traube, D.E. and McKay, M.M. (2007) Understanding the African American Research Experience (KAARE): Implications for HIV Prevention. Social Work in Mental Health, 5, 295-312. https://doi.org/10.1300/J200v05n03_03
[18]
Pottick, K.J. and Lerman, P. (1991) Maximizing Survey Response Rates for Hard-to-Reach Inner-City Populations. Social Science Quarterly, 72, 172-180.
[19]
Merry, S.E. (1981) Urban Danger: Life in a Neighborhood of Strangers. Temple University Press, Philadelphia, PA.
[20]
Ross, C.E. and Jang, S.J. (2000) Neighborhood Disorder, Fear, and Mistrust: The Buffering Role of Social Ties with Neighbors. American Journal of Community Psychology, 28, 401-420. https://doi.org/10.1023/A:1005137713332
[21]
Yancey, A.K., Ortega, A.N. and Kumanyika, S.K. (2006) Effective Recruitment and Retention of Minority Research Participants. Annual Review of Public Health, 27, 1-28. https://doi.org/10.1146/annurev.publhealth.27.021405.102113
[22]
Sampson, R.J., Morenoff, J.D. and Gannon-Rowley, T. (2002) Assessing “Neighborhood Effects”: Social Processes and New Directions in Research. Annual Review of Sociology, 28, 443-478. https://doi.org/10.1146/annurev.soc.28.110601.141114
[23]
Peterson, R.D., Krivo, L.J. and Harris, M.A. (2000) Disadvantage and Neighborhood Violent Crime: Do Local Institutions Matter? Journal of Research in Crime and Delinquency, 37, 31-63. https://doi.org/10.1177/0022427800037001002
[24]
Veysey, B.M. and Messner, S.F. (1999) Further Testing of Social Disorganization Theory: An Elaboration of Sampson and Groves’s “Community Structure and Crime”. Journal of Research in Crime and Delinquency, 36, 156-174.
https://doi.org/10.1177/0022427899036002002
[25]
Odierna, D.H. and Schmidt, L.A. (2009) The Effects of Failing to Include Hard-to-Reach Respondents in Longitudinal Surveys. American Journal of Public Health, 99, 1515-1521. https://doi.org/10.2105/AJPH.2007.111138
[26]
Weitzman, B.C., Guttmacher, S., Weinberg, S. and Kapadia, F. (2003) Low Response Rate Schools in Surveys of Adolescent Risk Taking Behaviours: Possible Biases, Possible Solutions. Journal of Epidemiology and Community Health, 57, 63-67. https://doi.org/10.1136/jech.57.1.63
[27]
National Institutes of Health: Office of Extramural Research (2001) NIH Policy and Guidelines on The Inclusion of Women and Minorities as Subjects in Clinical Research—Amended, October, 2001.
http://grants.nih.gov/grants/funding/women_min/guidelines_amended_10_2001.htm
[28]
Belone, L., Lucero, J.E., Duran, B., Tafoya, G., Baker, E.A., Chan, D., et al. (2016) Community-Based Participatory Research Conceptual Model: Community Partner Consultation and Face Validity. Qualitative Health Research, 26, 117-135.
https://doi.org/10.1177/1049732314557084
[29]
Hacker, K. (2013) Community-Based Participatory Research. Sage Publications, Thousand Oaks.
[30]
Oetzel, J.G., Zhou, C., Duran, B., Pearson, C., Magarati, M., Lucero, J., et al. (2015) Establishing the Psychometric Properties of Constructs in a Community-Based Participatory Research Conceptual Model. American Journal of Health Promotion, 29, e188-e202.
[31]
Chevalier, J.M. and Buckles, D. (2013) Participatory Action Research: Theory and Methods for Engaged Inquiry. Routledge, Abingdon-on-Thames.
Marcellus, L. (2004) Are We Missing Anything? Pursuing Research on Attrition. CJNR (Canadian Journal of Nursing Research), 36, 82-98.
[34]
Collins, L.M., Schafer, J.L. and Kam, C. (2001) A Comparison of Inclusive and Restrictive Strategies in Modern Missing Data Procedures. Psychological Methods, 6, 330-351. https://doi.org/10.1037/1082-989X.6.4.330
[35]
Schafer, J.L. and Graham, J.W. (2002) Missing Data: Our View of the State of the Art. Psychological Methods, 7, 147-177. https://doi.org/10.1037/1082-989X.7.2.147
[36]
Daniels, M.J. and Hogan, J.W. (2008) Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. Taylor & Francis, Boca Raton, FL.
[37]
Molenberghs, G. and Kenward, M. (2007) Missing Data in Clinical Studies. Wiley, West Sussex, England. https://doi.org/10.1002/9780470510445
[38]
Guan, Z. and Qin, J. (2017) Empirical Likelihood Method for Non-Ignorable Missing Data Problems. Lifetime Data Analysis, 23, 113-135.
https://doi.org/10.1007/s10985-016-9381-0
[39]
Kim, J.K. and Yu, C.L. (2011) A Semiparametric Estimation of Mean Functionals with Nonignorable Missing Data. Journal of the American Statistics Association, 106, 157-165. https://doi.org/10.1198/jasa.2011.tm10104
[40]
Scharfstein, D.O., Rotnitzky, A. and Robins, J.M. (1999) Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models. Journal of the American Statistical Association, 94, 1096-1146.
https://doi.org/10.1080/01621459.1999.10473862
[41]
Bolland, J.M., Lian, B.E. and Formichella, C.M. (2005) The Origins of Hopelessness among Inner-City African-American Adolescents. American Journal of Community Psychology, 36, 293-305. https://doi.org/10.1007/s10464-005-8627-x
[42]
King, D.W., Taft, C., King, L.A., Hammond, C. and Stone, E.R. (2006) Directionality of the Association between Social Support and Posttraumatic Stress Disorder: A Longitudinal Investigation. Journal of Applied Social Psychology, 36, 2980-2992.
https://doi.org/10.1111/j.0021-9029.2006.00138.x
[43]
McAuley, E., Elavsky, S., Motl, R.W., Konopack, J.F., Hu, L. and Marquez, D.X. (2005) Physical Activity, Self-Efficacy, and Self-Esteem: Longitudinal Relationships in Older Adults. Journal of Gerontology, 60, 268-275.
https://doi.org/10.1093/geronb/60.5.P268
[44]
Peugh, J.L. and Enders, C.K. (2004) Missing Data in Educational Research: A Review of Reporting Practices and Suggestions for Improvement. Review of Educational Research, 74, 525-556. https://doi.org/10.3102/00346543074004525
[45]
Rebellon, C.J., Manasse, M.E., Van Gundy, K.T. and Cohn, E.S. (2014) Rationalizing Delinquency: A Longitudinal Test of the Reciprocal Relationship between Delinquent Attitudes and Behavior. Social Psychology Quarterly, 77, 361-386.
https://doi.org/10.1177/0190272514546066
[46]
Rahn, W.M. and Transue, J.E. (1998) Social Trust and Value Change: The Decline of Social Capital in American Youth, 1976-1995. Political Psychology, 19, 545-565.
https://doi.org/10.1111/0162-895X.00117
[47]
U.S. Census Bureau (2012) 2000 Census. https://www.census.gov/
[48]
U.S. Census Bureau (2012) 1990 Census. https://www.census.gov/
[49]
Bolland, K.A., Bolland, J.M., Tomek, S., Devereaux, R.S., Mrug, S. and Wimberly, J.C. (2016) Trajectories of Adolescent Alcohol Use by Gender and Early Initiation Status. Youth & Society, 48, 3-32. https://doi.org/10.1177/0044118X13475639
[50]
Bolland, J.M. (2007) Mobile Youth Survey Overview. Unpublished Manuscript, University of Alabama at Birmingham, Birmingham.
[51]
U.S. Census Bureau (2015) Table 8.
https://www.census.gov/topics/education/school-enrollment.html
[52]
Redford, J., Battle, D. and Bielick, S. (2016) Homeschooling in the United States: 2012 (NCES 2016-096). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington DC.
[53]
Freudenberg, N. and Ruglis, J. (2007) Reframing School Dropout as a Public Health Issue. Preventing Chronic Disease, 4, A107.
[54]
Townsend, L., Flisher, A.J. and King, G. (2007) A Systematic Review of the Relationship between High School Dropout and Substance Use. Clinical Child and Family Psychology Review, 10, 295-317. https://doi.org/10.1007/s10567-007-0023-7
[55]
Chen, D., Drabick, D.A. and Burgers, D.E. (2015) A Developmental Perspective on Peer Rejection, Deviant Peer Affiliation, and Conduct Problems among Youth. Child Psychiatry & Human Development, 46, 823-838.
https://doi.org/10.1007/s10578-014-0522-y
[56]
McGee, L. and Newcomb, M.D. (1992) General Deviance Syndrome: Expanded Hierarchical Evaluations at Four Ages from Early Adolescence to Adulthood. Journal of Consulting and Clinical Psychology, 60, 766-776.
https://doi.org/10.1037/0022-006X.60.5.766
[57]
Liang, K.Y. and Zeger, S.L. (1986) Longitudinal Data Analysis Using Generalized Linear Models. Biometrika, 73, 13-22. https://doi.org/10.1093/biomet/73.1.13
[58]
Diggle, P., Heagerty, P., Liang, K. and Zeger, S. (2002) Analysis of Longitudinal Data. Oxford University Press, Oxford, England.
[59]
Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences. 2nd Edition, Lawrence Erlbaum Associates, Inc., Hillsdale, NJ.
[60]
Bor, W., Najman, J., O’Callaghan, M., Williams, G.M. and Anstey, K. (2001) Aggression and the Development of Delinquent Behavior in Children. Australian Institute of Criminology Trends and Issues in Crime and Criminal Justice, 207, 1-6.
[61]
Buhi, E.R., Goodson, P. and Neilands, T.B. (2008) Out of Sight, Not Out of Mind: Strategies for Handling Missing Data. American Journal of Health Behavior, 32, 83-92. https://doi.org/10.5993/AJHB.32.1.8
[62]
Teitler, J.O., Reichman, N.E. and Sprachman, S. (2003) Costs and Benefits of Improving Response Rates for a Hard-to-Reach Population. The Public Opinion Quarterly, 67, 126-138. https://doi.org/10.1086/346011
[63]
Way, N. and Robinson, M.G. (2003) A Longitudinal Study of the Effects of Family, Friends, and School Experiences on the Psychological Adjustment of Ethnic Minority, Low-SES Adolescents. Journal of Adolescent Research, 18, 324-346.
https://doi.org/10.1177/0743558403018004001
[64]
Groves, R.M. (2006) Nonresponse Rates and Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 646-675.
[65]
Groves, R.M. and Peytcheva, E. (2008) The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis. Public Opinion Quarterly, 72, 167-189.
https://doi.org/10.1093/poq/nfn011
[66]
Lennon, M.C., Clark, W.A. and Joshi, H. (2016) Residential Mobility and Wellbeing: Exploring Children’s Living Situations and Their Implications for Housing Policy. Longitudinal and Lifecourse Studies, 7, 197-200.
https://doi.org/10.14301/llcs.v7i3.393
[67]
Rosenblatt, P. and DeLuca, S. (2012) “We Don’t Live Outside, We Live in Here”: Neighborhood and Residential Mobility Decisions Among Low-Income Families. City & Community, 11, 254-284. https://doi.org/10.1111/j.1540-6040.2012.01413.x
[68]
Bolland, A.C. (2012) Representativeness Two Ways: An Assessment of Representativeness and Missing Data Mechanisms in a Study of an At-Risk Population. Doctoral Dissertation, The University of Alabama, Tuscaloosa.
[69]
Von Stumm, S. and Plomin, R. (2015) Socioeconomic Status and the Growth of Intelligence from Infancy through Adolescence. Intelligence, 48, 30-36.
[70]
de Winter, A.F., Visser, L., Verhulst, F.C., Vollebergh, W.A. and Reijneveld, S.A. (2016) Longitudinal Patterns and Predictors of Multiple Health Risk Behaviors among Adolescents: The TRAILS Study. Preventive Medicine, 84, 76-82.
https://doi.org/10.1016/j.ypmed.2015.11.028
[71]
Evans, G.W., Li, D. and Whipple, S.S. (2013) Cumulative Risk and Child Development. Psychological Bulletin, 139, 1342-1396. https://doi.org/10.1037/a0031808
[72]
Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., et al. (2014) Reaching the Hard-to-Reach: A Systematic Review of Strategies for Improving Health and Medical Research with Socially Disadvantaged Groups. BMC Medical Research Methodology, 14, 42. https://doi.org/10.1186/1471-2288-14-42
[73]
Faugier, J. and Sargeant, M. (1997) Sampling Hard to Reach Populations. Journal of Advanced Nursing, 26, 790-797. https://doi.org/10.1046/j.1365-2648.1997.00371.x
