Analysis and Reporting Resources for the End Commercial Tobacco Campaign Observation Surveys

Welcome to the resource page on analysis and reporting!

This page contains information and resources for analyzing and reporting on the End Commercial Tobacco Campaign Observation Surveys (MUH, Retail, Parks, and Sidewalks). For additional assistance, please contact tcecTA@phmail.ucdavis.edu

Expectations

Each LLA is expected to do their own data cleaning, analysis, and reporting of the End Commercial Tobacco Campaign Observation Surveys. While data collection for these observations must be completed by June 30, the analysis and report can be submitted in the next progress report period.

Each LLA Lead Contact can expect to receive their raw data, codebook, and data analysis guidelines via email. The data analysis guidelines, codebooks, and other resources are also listed on this page below. If your LLA did not receive these documents, please contact your TCEC representative or tcecTA@phmail.ucdavis.edu

If your LLA finished data collection early, we asked that you do not widely share your results so that it does not disrupt other LLAs collect data up to June 30th. Sharing with your coalition, internal team, and with key stakeholders is reasonable. Please do not issue press releases or disseminate widely. If you need to publicly release data prior to June 30th, please contact Catherine at czdizon@ucdavis.edu.

Each LLA will be analyzing their own data. While CTCP will be doing some overall analysis, they will not be creating them for each LLA.

Recordings from the webinars on data analysis in Excel that uses the ECTC data as examples is listed in the additional resources links below. For those that use statistical data analysis software, there are some sample codes in the guidelines linked below as well as weighting variables that should be used to perform weighted calculations.

Remember that all data must be reported in aggregate. Do NOT report individual results, names, or addresses. E.g., do NOT display individual results on a map. Do NOT report results for areas with fewer than 5 MUH or retailers.

Data Analysis Guidelines

Please be sure to thoroughly read the data analysis guidelines relevant to your LLA's workplan. Each document includes information about where to access important resources, who to contact with questions, sampling, weights, analysis suggestions, and sample SAS codes, outputs, and interpretations. It contains a lot of helpful, detailed information that is intended to provide a starting point for your local analyses. We highly recommend that anyone analyzing the data carefully read the guidelines before working with the data.

2022 guidelines:

2024 guidelines: CTPP is updating the guidelines for 2024. You can expect to see those here in early July.

Codebooks and Instruments

Relevant codebooks were emailed to each LLA's Lead Contact along with their raw data. The files contain separate tabs for retail, MUH, park, and sidewalk variables.

The data collection instruments for all four observation activities are listed below. Please note that the protocol for the End Commercial Tobacco Campaign is to collect data on mobile devices. Do NOT collect data on paper forms.

Requirements for Sharing Data

Each LLA is responsible for processing requests to access their data. Below is a Data Request Tracking Sheet that helps track when an LLA receives such requests. Also included is a Sample Data Request Form that gives an idea of how one might request an LLA's data.

When an LLA shares their observation data, it should be accompanied by the Sampling Methodology, relevant data analysis guidelines (MUH | Parks | Sidewalks | Retail),

Anyone that has access to the data must only report data in aggregate. Do NOT report individual results, names, or addresses. E.g., do NOT display individual results on a map. Do NOT report results for areas with fewer than 5 MUH or retailers.

Links to Additional Resources

Analysis Q/A

Q: What is due and by when? What do I add to the progress report?

A: The deadline to collect all ECTC Observation data is June 30th
The deadline to submit the instrument and training materials is July 31st (with your July progress report in OTIS)
The deadline to submit the summary report is Jan. 31st (with your January progress report in OTIS)

For most LLAs, you will need to submit the blank observation data collection instrument(s) and an observation summary report. For some LLAs that have data collector trainings in their workplan, data collection training materials will also need to be submitted.

The deadline to submit the blank data collection instrument and the data collection materials is July 31st, with the rest of your progress report deliverables. LLAs have a bit more time to analyze the data and write the summary report. Because of the way progress reports are submitted every 6 months, the next time LLAs submit deliverables in OTIS is Jan. 31st. Please do not just wait until then to put this data into action! Be sure to analyze and use sure to use this amazing treasure trove of data asap. The reason why there is extra time to submit the summary report compared to other observations in a workplan, is that the TCEC team needed to prepare your data and CTPP will be releasing an updated data analysis guidelines in the coming weeks. The state will do some overall analysis, but they won’t be doing individual, county or jurisdiction level analysis. Each LLA is expected to conduct their own analysis, reporting, and dissemination.

The blank data collection instruments can be downloaded here. Download the relevant instrument (retail, MUH, or park/sidewalk) and submit as a clearly labeled file with your specific activity number and the words, “Observation Instrument.” If desired, you can add more information such as the name of the jurisdiction or dates if you prefer. Here's an example: "2-E-3 Summerland MUH Observation Instrument"
The data collection training materials can be downloaded here. Download the relevant training materials (intro, retail, MUH, or park/sidewalk, quiz, etc.) and submit as a clearly labeled file with your specific activity number and the words, "Data Collection Training Materials." If desired, you can add more information such as the name of the jurisdiction or dates if you prefer. Here's an example: "2-E-3 Summerland MUH Data Collection Training Materials" Or if you created your own materials, you should submit your training materials.
The observation summary report is produced by each LLA.

What do I do with the stores that didn't sell any tobacco?
LLAs should keep the record in the dataset as the store will still be observed in the next wave. This is because the statewide evaluation is based on stores that were licensed by CDTFA, regardless of if the store had tobacco for sale or not at the time of data collection. We also want to be able to track if the proportion of stores selling tobacco changes in the next wave of data collection.

When LLAs analyze this first round of data, it is recommended that LLAs analyze it both ways: one with the stores that did not sell tobacco and one without stores that sell tobacco. LLAs can choose which analysis is best for them or use both. The most important part is just to make sure the to state the findings correctly. For example, if an LLA is looking at availability of flavored tobacco products and their analytic sample does contain stores that no longer sell tobacco, the universe or denominator should be of stores that were licensed to sell tobacco (i.e., X% of stores that were licensed to sell tobacco sold flavored tobacco). If their analytic sample does not contain any stores that do not sell tobacco, the universe/denominator is of stores that sell tobacco (i.e., X% of stores that sell tobacco sold flavored tobacco).
What is my sample size? Is it the number of stores that sold tobacco? Is it the number of stores I observed? Is it the whole list of stores from CTCP?
It depends! You may want to report on the responses from questions 10-17 or if you're interested in describing all stores you observed, your sample size can be based on the answer to question 5 being, "Yes, I can." If you're reporting on the type of products sold at tobacco stores, your sample size is based on the answer to question 18 being, "Vaping products," OR "Any other tobacco."

For more details, please see the Data Analysis Guidelines for retail observation
The confidence interval is really wide, it even includes a negative value. Do I still report it?
In many cases, a wide confidence interval is due to a very low sample size. It may not make sense to do a confidence interval as the margin of error is likely going to be very high, resulting in an even wider confidence interval. Instead, you could make note of that limitation in the report.
When do we get the weighting variables?
Weighting variables are available by messaging tcecTA@phmail.ucdavis.edu Please use the subject line: "ECTC Weighting Variables." Weighting variables are only necessary for LLAs with a lot of missing retail or park data. Weighting variables are not necessary nor available for MUH or sidewalk data.

Using weighting variables is also only appropriate when using SAS, SPSS, R, or other statistical analysis software. For those using Excel to analyze data, it is not recommended to use the weighting variables.

Click here for the SAS or SPSS code to combine the weighting variables with your data.
What tips do you have for analyzing data in Excel?
Check out our videos:
Using the =COUNTIF function
Conditional formatting, icons, and bars
Why it's important to lock cells