How much does pre-trial testing influence complex intervention trials and would more testing make any difference? An email survey

Email survey of the authors of complex intervention trials published in seven major journals in 2004.72% (50/69) of eligible authors replied. Eight authors did not consider their interventions to be complex. The majority of respondents' complex interventions were tested (34/42): some extensively. Conversely, only 17 of the 34 published reports describing these trials mention testing. Two-thirds (22/34) of those testing their interventions did not believe that more or different testing would have produced a more effective intervention. 31% (13/42) of all authors did believe further testing would have led to improvements. Five respondents mentioned a lack of funding as a reason for not doing more testing.Complex interventions are generally tested prior to their evaluation in a full-scale trial, although the amount of testing varies. Testing is often not described in trial reports, which makes it hard to judge whether a trial result could be improved with a better intervention, or whether further work with a different intervention is required.In 2000 the UK Medical Research Council published a structure for evaluating complex interventions, which proposed that interventions should be modelled and then tested prior to a full-scale trial [1]. The pre-trial work could be expected to provide a more realistic estimate of the likely effect of the intervention and inform decisions regarding intervention design and delivery. In principle, such work may help to avoid wasting resources evaluating ineffective interventions because rigorous development and testing will reject poor interventions and unfeasible trials before they reach full-scale evaluation [2,3].The aim of the current study was twofold. Firstly, we wanted to survey the estimates of treatment effect and recruitment given in trial reports and compare them with those actually achieved, together with any mention of piloting or testing. Secondly, we wanted to ask the authors of these trial reports whether testing did in