Analysis and Reporting Resources for the End Commercial Tobacco Campaign Observation Surveys

Welcome to the resource page on analysis and reporting!

This page contains information and resources for analyzing and reporting on the End Commercial Tobacco Campaign Observation Surveys (MUH, Retail, Parks, and Sidewalks).  For additional assistance, please contact tcecTA@phmail.ucdavis.edu

Expectations

Each LLA is expected to do their own data cleaning, analysis, and reporting of the End Commercial Tobacco Campaign Observation Surveys. While data collection for these observations must be completed by June 30, 2022, the analysis and report can be submitted in the next progress report period.

Each LLA Lead Contact can expect to receive their raw data, codebook, and data analysis guidelines via email.  The data analysis guidelines, codebooks, and other resources are also listed on this page below.  If your LLA did not receive these documents, please contact your TCEC representative or tcecTA@phmail.ucdavis.edu

If your LLA finished data collection early, we asked that you do not widely share your results so that it does not disrupt other LLAs collect data up to June 30th.  Sharing with your coalition, internal team, and with key stakeholders is reasonable.  Please do not issue press releases or disseminate widely.  If you need to publicly release data prior to June 30th, please contact Catherine at czdizon@ucdavis.edu.

Each LLA will be analyzing their own data.  While CTCP will be doing some overall analysis, they will not be creating them for each LLA. 

Recordings from the webinars on data analysis in Excel that uses the ECTC data as examples is listed in the additional resources links below.  For those that use statistical data analysis software, there are some sample codes in the guidelines linked below as well as weighting variables that should be used to perform weighted calculations.

Remember that all data must be reported in aggregate. Do NOT report individual results, names, or addresses. E.g., do NOT display individual results on a map. Do NOT report results for areas with fewer than 5 MUH or retailers.

Data Analysis Guidelines

Please be sure to thoroughly read the data analysis guidelines relevant to your LLA's workplan.  Each document includes information about where to access important resources, who to contact with questions, sampling, weights, analysis suggestions, and sample SAS codes, outputs, and interpretations.  It contains a lot of helpful, detailed information that is intended to provide a starting point for your local analyses.  We highly recommend that anyone analyzing the data carefully read the guidelines before working with the data.

Remember that all data must be reported in aggregate. Do NOT report individual results, names, or addresses. E.g., do NOT display individual results on a map. Do NOT report results for areas with fewer than 5 MUH or retailers.

Codebooks

Relevant codebooks were emailed to each LLA's Lead Contact along with their raw data.  All four codebooks are also available here with a separate tab for retail, MUH, park, and sidewalk variables.  

After July, LLAs may request to receive their data with string or text variables instead of the default code variables.  

Requirements for Sharing Data

Each LLA is responsible for processing requests to access their data.  Below is a Data Request Tracking Sheet that helps track when an LLA receives such requests.  Also included is a Sample Data Request Form that gives an idea of how one might request an LLA's data.  

When an LLA shares their observation data, it should be accompanied by the Sampling Methodology, relevant data analysis guidelines (MUH | Parks | Sidewalks | Retail), 

Anyone that has access to the data must only report data in aggregate.  Do NOT report individual results, names, or addresses.  E.g., do NOT display individual results on a map. Do NOT report results for areas with fewer than 5 MUH or retailers.

Links to Additional Resources

Analysis Q/A

  • What do I do with the stores that didn't sell any tobacco?
  • LLAs should keep the record in the dataset as the store will still be observed in the next wave. This is because the statewide evaluation is based on stores that were licensed by CDTFA, regardless of if the store had tobacco for sale or not at the time of data collection.  We also want to be able to track if the proportion of stores selling tobacco changes in the next wave of data collection.

    When LLAs analyze this first round of data, it is recommended that LLAs analyze it both ways: one with the stores that did not sell tobacco and one without stores that sell tobacco.  LLAs can choose which analysis is best for them or use both.  The most important part is just to make sure the to state the findings correctly.  For example, if an LLA is looking at availability of flavored tobacco products and their analytic sample does contain stores that no longer sell tobacco, the universe or denominator should be of stores that were licensed to sell tobacco (i.e., X% of stores that were licensed to sell tobacco sold flavored tobacco).  If their analytic sample does not contain any stores that do not sell tobacco, the universe/denominator is of stores that sell tobacco (i.e., X% of stores that sell tobacco sold flavored tobacco).
  • What is my sample size?  Is it the number of stores that sold tobacco?  Is it the number of stores I observed?  Is it the whole list of stores from CTCP?  
  • It depends!  You may want to report on the responses from questions 10-17 or if you're interested in describing all stores you observed, your sample size can be based on the answer to question 5 being, "Yes, I can."  If you're reporting on the type of products sold at tobacco stores, your sample size is based on the answer to question 18 being, "Vaping products," OR "Any other tobacco." 

    For more details, please see the Data Analysis Guidelines for retail observation
  • The confidence interval is really wide, it even includes a negative value.  Do I still report it?
  • In many cases, a wide confidence interval is due to a very low sample size.  It may not make sense to do a confidence interval as the margin of error is likely going to be very high, resulting in an even wider confidence interval.  Instead, you could make note of that limitation in the report. 
  • When do we get the weighting variables?
  • Weighting variables are available by messaging tcecTA@phmail.ucdavis.edu  Please use the subject line: "ECTC Weighting Variables."  Weighting variables are only necessary for LLAs with a lot of missing retail or park data.  Weighting variables are not necessary nor available for MUH or sidewalk data.

    Using weighting variables is also only appropriate when using SAS, SPSS, R, or other statistical analysis software.  For those using Excel to analyze data, it is not recommended to use the weighting variables.

    Click here for the SAS or SPSS code to combine the weighting variables with your data.
  • What tips do you have for analyzing data in Excel? 
  • Check out our videos:
    Using the =COUNTIF function
    Conditional formatting, icons, and bars
    Why it's important to lock cells