Marlene Schwartz, Ph.D., is deputy director of the Rudd Center for Food Policy and Obesity at Yale University. In June 2006, Dr. Schwartz received a Healthy Research Round 1 grant to evaluate the effectiveness of school wellness policies in Connecticut. One of the goals of her research was to develop and validate a coding system that could be used to quantitatively evaluate school wellness policies. To arrive at a reliable tool, Dr. Schwartz collaborated with other Healthy Eating Research grantees pursuing similar research goals in other states. The product of the collaboration was a rigorous, statistically validated coding tool, the details of which were published in the Journal of the American Dietetic Association in July 2009.  

Q. What led you and the working group of other Healthy Eating Research grantees to pursue the development of a coding system?

Schwartz: The grantees involved in this collaborative were funded to evaluate the impact of school wellness policies in their respective states. We soon realized that we needed a way to evaluate these policies in a quantitative way. That need inspired the creation of the coding tool. But developing a robust, validated evaluation tool was quite complicated, which is why we needed to collaborate with other researchers.

Q. In addition to the Rudd Center, what other institutions contributed to the coding system?

Schwartz: The collaboration included researchers from Penn State University (Elaine McDonnell, M.S., R.D.; Claudia Probart, Ph.D., R.D.), the University of Washington (Anne Lund, R.D., M.P.H.; H. Mollie Grow, M.D., M.P.H.), and the University of Minnesota (Leslie Lytle, Ph.D., R.D.; Anne Samuelson, M.P.H.).

Q. Have other states used this coding system?  

Schwartz: Since we created the coding system and word has gotten out about it, we've been contacted by many other states. It has reportedly been used in Maine, Delaware, Florida, North Carolina, Michigan, West Virginia and Massachusetts. Some people are looking to apply the tool the way we did, to evaluate and compare policies across a single state. Others are using it for specific projects, to evaluate a few district policies as part of a more in-depth study of the school environment.

Q. Why was it critical to establish a coding system for evaluating the quality of school wellness policies? How were school wellness policies previously evaluated, and why was that approach not as effective?

Schwartz: The materials that were already available were not adequate to provide a quantitative score. The existing measures were either broad categorizations of policy strength or self-assessments. For example, the American Heart Association's and Clinton Foundation's Alliance for a Healthier Generation assigned wellness policies to gold, silver or bronze levels, which was not detailed enough for our purposes. Action for Healthy Kids and the Center for Science in the Public Interest presented ideas about what you could put in your policy, but didn't assign a score to different components. What we needed was a measure that could give us a meaningful score between 0 and 100 to assess the strength of the policy. We wanted something precise and with the potential to be sensitive to small changes. Our paper showed that the coding system was a methodologically rigorous evaluation tool. We showed that the coding system was reliable, and we were able to show that the ideas behind the tool held up psychometrically. This means that the way we clustered the items together into subscales, such as nutrition education, physical education, and standards for foods sold outside the school meals, reflected the way that policies were actually written. Policies tended to score consistently within each of those domains.  

Q. What were some of the challenges you encountered during the research-either in creating the coding system or in gathering the data for evaluation?

Schwartz: The federal language mandating school wellness policies is very vague - it says policies need to cover certain topics, but there is very little direction on exactly what they need to say. The sample policies that people devised on the basis of those government definitions were also highly variable. So it was really difficult trying to keep the categories down to a manageable number. We went through hundreds of items until we landed on the 96 we included in the published paper. Now we are even trying to get it down lower.

Q. What is the most important or surprising thing you learned from the study?

Schwartz: The most surprising thing was learning how many policies were poorly written and did not have ‘teeth.' I had been under the impression that if you took the time to write these policies and regulations, they would have specific goals and objectives. In reality, though, many districts put down their philosophical goals but really didn't promise much of anything. For example, they would say, "We discourage teachers from giving children unhealthy food as a reward." What does "discourage" mean? As I was evaluating these policies, the way I kept thinking about it is that if I were a parent in this school district, would the policy definitely tell me what was and was not allowed?

Q. What are some of the policy opportunities that this research offers/affords?

Schwartz: This research raises the question of how much one can rely on local control. Most districts across the country have very weak policies. I think there is a need for strong federal and state laws to push districts to strengthen what they are doing.  

Q. What would you like to see materialize as an end result of your work?

Schwartz: It would be interesting to see if different parts of the country are able to use this as a tool to distinguish between strong policies and weak ones. In Connecticut, we coded all of our districts' policies and then sent report cards back to the districts. We told them, this is your score, this is how you compare with the state average, this is how you compare with other districts, and here are your strengths and weaknesses. Because of that, we have seen greater motivation for school districts to improve their score. I'd like to see this happening nationally as well-have states code their districts and give a report card to each district so they can create stronger policies.