by Jayveeritz Bautista

As the statewide evaluation technical assistance provider for California’s tobacco control programs, the Tobacco Control Evaluation Center maintains a database of data collection instruments previously used by local projects. A key piece of this puzzle is screening and selecting instruments for quality.

Alongside Barry Hill from the Center for Healthcare Policy and Research and undergraduates from UC Davis, TCEC began validating instruments from the latest round of Final Evaluation Reports in April of this year. Our purpose for this project was to improve how tobacco control surveys are worded and structured in hopes of helping people understand and respond to the questions as intended, and collect valid, reliable information.

To meet our goals, we conducted our instrument testing with initial phone screenings to determine eligibility, followed by in-person interviews that included probing and debriefing questions.  We adapted this method from a 2003 U.S. Census Bureau report by Hughes and Mingay.

Each instrument had a variety of questions related to a topic of interest. For our first round, the topics included:

• tobacco use in multi-unit housing
• outdoor recreational areas
• zoning in youth-sensitive areas
• electronic smoking devices

We removed repetitive or inessential questions and made minor changes to questions for clarification. In total, each instrument consisted of about 20 questions, excluding probing, debriefing, and demographics questions.

Within the next few months, we were able to finalize all our surveys, create all essential tools for recruitment, and gather all necessary supplies for the interviews. In early June, recruitment began with the posting of flyers all around Davis and several social media pages. Within less than a week, we had our first participant. Each survey took around 15 minutes, but the amount of time heavily relied on the participants’ willingness to provide their cognitive thinking process. Thus, for participants taking all four of our surveys, it took between one to two hours. For every interview, we translated any discovered significant areas of improvement for our questions to the final version of our instrument.

After finishing all the interviews, the following month consisted of data analysis and organization for our final report. The four instruments used in the project have been uploaded onto the TCEC instrument database.

However, this is not where TCEC’s instrument validation project ends. Currently, as of September 2018, we are finalizing more surveys consisting of different topics, such as tobacco use in LGBTQ communities, home exposure to tobacco smoke, and flavored tobacco products. Recruitment for a new round of instrument validation is already underway. Be sure to check back with TCEC to see more of our validated instruments soon!

If you need help searching the database or developing your own instrument, give us a call at 530-752-9951, or email tobaccoeval@ucdavis.edu.