by Jorge Andrews & Andersen Yang

Every August, the Claremont Evaluation Center (CEC) at Claremont Graduate University offers a series of professional development courses, which introduce several topics related to evaluation. This year Andersen Yang and I flew down to Southern California to get our learning on! Here are some of our thoughts from CEC evaluation courses. If you are interested in taking courses at Claremont Graduate University or would like more information about Claremont Evaluation Center, visit their website here.

Jorge’s Claremont Experience:

This year, CEC offered a broader and fresh offering of courses. Of course, there were the staples that introduce evaluation and survey development, but there were some other exciting offerings, such as a course presented by Katrina Bledsoe on “Culturally Responsive Evaluation,” and “Science of Wellbeing” presented by Stewart Donaldson.

Culturally Responsive Evaluation

The first course I attended was Culturally Responsive Evaluation, presented by Katrina Bledsoe. The course was amazing. TCEC has been emphasizing and developing resources on incorporating cultural humility into tobacco control evaluation work, and this Claremont course really validated much of our work and research on the topic. The underlying theme of the course was self-reflection and love for humanity.

The use of the word “love” took me by surprise, since in academia and even in evaluation we tend sanitize subjective terms. It might be due to this that we lose sight of the reason we do our work. What drew me to this work is social justice; I hope to help make the world a more equitable place. Social justice and equity are a big part of why we should include culturally responsive evaluation into our work.

Including culturally responsive evaluation and cultural humility in our work requires us to develop strong relationships. It is by growing our community roots that we can build healthy communities, which promotes understanding and positive development. Through evaluation, we have opportunities to include the community’s voices, and facilitate long term sustainability.  

Science of Well-Being

This course was a based on positive psychology, a field of study pioneered by Claremont Graduate University and presenter/professor Stewart Donaldson and Saeideh Heshmati. The underlying message of the course was that to make the world a better place we should strive for happiness within ourselves and those around us. Sounds easy enough, right? Happiness is not a passive thing, however, but rather something that requires effort. I must admit, I am skeptical about examining something as abstract as happiness through simple-to-measure scientific exercises. However, Dr. Donaldson and Dr. Heshmati shared existing research that explains happiness as positive moments which bring satisfaction, and how the sum of these moments is what makes up a happy and healthy individual.

It was interesting to see an immaterial topic discussed in an academic way. It just goes to show that academic curiosity can encompass a wide range of topics. You may be asking, “How is this relevant to evaluation work?” I would argue that by incorporating some of these principles into our everyday work, we can make things better for ourselves and others. Simple things like clear and compassionate communication with our colleagues can promote a healthier and more productive work environment. We can all benefit from that.

Happiness requires work, but if we put in the effort to encourage wellness, the results are more productive and positive individuals. Through positive psychology and promoting well-being in ourselves and those around us, we are likely to spread that positivity to others with whom we engage, including the communities we serve. I don’t see a downside to trying to incorporate some of these lessons into our everyday work and personal lives, do you?

Final Thoughts

I had not attended the CEC courses in a few years, because I felt that I had gotten the most out of the courses offered in the past, but it was very uplifting to see the new offerings that CEC had for 2019. I was pleasantly surprised with the courses I attended, I walked out inspired and motivated to apply some of the lessons learned. It was also nice to run into a few colleagues from tobacco control, Addison Saenz and Monika Gracian from San Bernardino, California Health Collaborative. Getting to know Addison and Monika was an unexpected bonus to this trip, and I hope to get to know more of you in the future!

Andersen’s Claremont Experience:  

I came into the CEC evaluation professional development series with little formal training in evaluation. At TCEC, we provide evaluation technical assistance to California tobacco control. I was eager to attend, because as a Master of Public Health student at UC Davis, I learned that evaluation is one of the 10 essential public health services. 

Qualitative Research Methods

The first class I took, “Qualitative Research Methods” was rather daunting during the classmate introductions. Most of them had their PhDs or were in high positions at their corporate jobs. But we all came in with the same mindset-- to gain more exposure to qualitative methods. Since most of us came from a quantitative background, the professor enthusiastically explained the differences between quantitative and qualitative methods.

Whereas quantitative methods target a specific objective truth through numbers, qualitative methods are constructing and interpreting reality through participant’s words. The interpretation of opinions and stories are captured through qualitative methods. I was stunned by how focus groups and interviews can also offer such rich data and insights when writing up an evaluation report. Having a strong bias towards statistics based on my background, I realize that stories and narratives can resonate more with people.

Quasi Experimental Research Methods

I was surprised by the small class size of the next course, “Quasi Experimental Research Methods.” Admittedly, the name does sound scary, but the professor broke the subject down, explaining why and how we use this research method. Quasi experimental methods differ from true experiments in that there is no random assignment of participants to the intervention or control group. This is more relevant to tobacco control evaluation, since experimental methods may not be feasible nor ethical. I enjoyed the professor’s research examples that accompanied each subtopic.

This workshop showed me how important this method can be when we can’t conduct a true experiment. For example, in tobacco control, we might want to evaluate whether a tobacco media campaign had any effect on a group of smokers. Though we typically have no way to smokers to a media intervention group or control group, we can still evaluate the impact of the intervention using a common quasi experimental method, known as the pre-and-post comparison test. 

Survey Research Methods 

The final day, I attended the “Survey Research Methods” workshop. Even though it was 9 AM Saturday class, I felt energetic to learn about developing surveys. Coincidentally enough, the professor brought so much energy and enthusiasm with him! Throughout the lecture, I kept daydreaming about my experiences with testing and refining surveys at TCEC. Yes, I did still pay attention! The professor could not have stressed more how important it is to pilot a survey before launching it into the public. Piloting a survey allows us to remove any blind errors that others may be more inclined to observe. It’s critical that the questions and response sets are well-worded, or else there will be a lot of complaints in the email inbox within a day of launching the survey. That will delay the data collection process and jeopardize the quality of the data. I was motivated to apply his tips and strategies to develop effective surveys in the future.

Final Thoughts

Overall, I had a great time at Claremont and the trip down to Southern California. I got to meet so many like-minded people in during my time at CEC. There was an array of professionals ranging from psychologists to market researchers to even CEOs of their own companies! Nonetheless, we were without a doubt unified in our pursuit of learning more about evaluation.