UC Berkeley

This dissertation presents a quasi-experimental study design to evaluate non-random\-ized, pre-existing community interventions not originally designed to gather rigorous data about sustainability and impact. The core components of the design include selection of a control group using propensity score matching with pre-intervention (baseline) secondary data and post-intervention follow-up in the field. The main advantages of the design include measurement of interventions implemented under actual field conditions (independent of scientific research), as well as the design's ability to gather information about the long term impacts and sustainability of interventions without years of costly prospective follow-up. Studies of non-randomized, pre-exisiting interventions must address threats to validity, principal among them: unmeasured confounding and informative censoring.

I outline the main strengths and weaknesses of the study design using simulation and empirical examples. I also apply the design in two sustainability field studies: a 3-year household water treatment and hygiene promotion intervention in rural Guatemala and a 5-year community led total sanitation, water supply and hygiene education intervention in rural India. In both studies, the design leads to samples of intervention and control groups with highly comparable baseline characteristics.

A principal finding of both field studies is poor initial impact and sustainability of the behavioral components of the interventions. In Guatemala, I find a small, five percentage point increase (8.7% vs. 3.3%) in the proportion of households that treat their water six months after the promotion intervention, but no differences in hygiene knowledge or practice, and no detectable differences in child health based on acute illness or growth. In India, I find a large, 33 percentage point increase (48% vs. 15%) in private toilet construction as a result of the intervention, but open defecation persists in 40% of households with functional private toilets. In India, diarrhea is rare in both intervention and control communities (1.8% over 14,259 child weeks), but most children show growth faltering by international standards (mean height-for-age Z-score: -1.98). Despite no signifiant differences in health between children living in intervention and control villages, I observe important non-health benefits: private toilets increase privacy and safety during defecation for women and girls by 28 percentage points (81% vs. 53%), and private water taps reduce water collection time by a median of 25 minutes per day relative to public taps (50 vs. 75 minutes). I also find that hardware improvements are highly sustainable up to five years after implementation with more than 94% of private toilets and 96% of private water taps in use during repeated visits over one year.

Studies of non-randomized, pre-existing interventions are a rapid, low-cost alternative to prospective intervention studies for evaluating intervention sustainability. The study design and methodology developed in this dissertation are applicable to evaluating a broad range of pre-existing, community interventions beyond the water, sanitation and hygiene sector.