Agreement Between Self-reports and Photos to Assess e-Cigarette Device and Liquid Characteristics in Wave 1 of the Vaping and Patterns of e-Cigarette Use Research Study: Web-Based Longitudinal Cohort Study

Background: e-Cigarette device and liquid characteristics are highly customizable; these characteristics impact nicotine delivery and exposure to toxic constituents. It is critical to understand optimal methods for measuring these characteristics to accurately assess their impacts on user behavior and health. Objective: To inform future survey development, we assessed the agreement between responses from survey participants (self-reports) and photos uploaded by participants and the quantity of usable data derived from each approach. Methods: Adult regular e-cigarette users (≥5 days per week) aged ≥21 years (N=1209) were asked questions about and submitted photos of their most used e-cigarette device (1209/1209, 100%) and liquid (1132/1209, 93.63%). Device variables assessed included brand, model, reusability, refillability, display, and adjustable power. Liquid variables included brand, flavor, nicotine concentration, nicotine formulation, and bottle size. For each variable, percentage agreement was calculated where self-report and photo data were available. Krippendorff α and intraclass correlation coefficient (ICC) were calculated for categorical and continuous variables, respectively. Results were stratified by device (disposable, reusable with disposable pods or cartridges, and reusable with refillable pods, cartridges, or tanks) and liquid (customized and noncustomized) type. The sample size for each calculation ranged from 3.89% (47/1209; model of disposable devices) to 95.12% (1150/1209; device reusability). Results: Percentage agreement between photos and self-reports was substantial to very high across device and liquid types for all variables except nicotine concentration. These results are consistent with Krippendorff α calculations, except where prevalence bias was suspected. ICC results for nicotine concentration and bottle size were lower than percentage agreement, likely because ICC accounts for the level of disagreement between values. Agreement varied by device and liquid type. For example, percentage agreement for device brand was higher among users of reusable devices (94%) than among users of disposable devices (75%). Low percentage agreement may result from poor participant knowledge of characteristics, user modifications of devices inconsistent with manufacturer-intended use, inaccurate or incomplete information on websites, or photo submissions that are not a participant's most used device or liquid. The number of excluded values (eg, self-report was “don't know” or no photo submitted) differed between self-reports and photos; for questions asked to participants, self-reports had more usable data than photos for all variables except device model and nicotine formulation. Conclusions: Photos and self-reports yield data of similar accuracy for most variables assessed in this study: device brand, device model, reusability, adjustable power, display, refillability, liquid brand, flavor, and bottle size. Self-reports provided more data for all variables except device model and nicotine formulation. Using these approaches simultaneously may optimize data quantity and quality. Future research should examine how to assess nicotine concentration and variables not included in this study (eg, wattage and resistance) and the resource requirements of these approaches.