The use of crowdsourcing for dietary self-monitoring: crowdsourced ratings of food pictures are comparable to ratings by trained observers Gabrielle M Turner-McGrievy, 1 Elina E Helander, 2 Kirsikka Kaipainen, 3 Jose Maria Perez-Macias, 2 Ilkka Korhonen 2,3 1 Health Promotion, Education, and Behavior, University of South Carolina, Arnold School of Public Health, Columbia, South Carolina, USA 2 Department of Signal Processing, Tampere University of Technology, Tampere, Finland 3 VTT Technical Research Centre of Finland, Tampere, Finland Correspondence to Dr Gabrielle Turner-McGrievy, Discovery I, 915 Greene Street, Room 529, Columbia, SC 29208, USA; brie@sc.edu Received 7 January 2014 Revised 17 June 2014 Accepted 18 June 2014 To cite: Turner-McGrievy GM, Helander EE, Kaipainen K, et al . J Am Med Inform Assoc Published Online First: [ please include Day Month Year] doi:10.1136/amiajnl-2014- 002636 ABSTRACT Objective Crowdsourcing dietary ratings for food photographs, which uses the input of several users to provide feedback, has potential to assist with dietary self-monitoring. Materials and methods This study assessed how closely crowdsourced ratings of foods and beverages contained in 450 pictures from the Eatery mobile app as rated by peer users (fellow Eatery app users) (n=5006 peers, mean 18.4 peer ratings/photo) using a simple ‘healthiness’ scale were related to the ratings of the same pictures by trained observers (raters). In addition, the foods and beverages present in each picture were categorized and the impact on the peer rating scale by food/beverage category was examined. Raters were trained to provide a ‘healthiness’ score using criteria from the 2010 US Dietary Guidelines. Results The average of all three raters’ scores was highly correlated with the peer healthiness score for all photos (r=0.88, p<0.001). Using a multivariate linear model (R 2 =0.73) to examine the association of peer healthiness scores with foods and beverages present in photos, peer ratings were in the hypothesized direction for both foods/beverages to increase and ones to limit. Photos with fruit, vegetables, whole grains, and legumes, nuts, and seeds (borderline at p=0.06) were all associated with higher peer healthiness scores, and processed foods (borderline at p=0.06), food from fast food restaurants, refined grains, red meat, cheese, savory snacks, sweets/desserts, and sugar-sweetened beverages were associated with lower peer healthiness scores. Conclusions The findings suggest that crowdsourcing holds potential to provide basic feedback on overall diet quality to users utilizing a low burden approach. INTRODUCTION Behavioral weight loss interventions are an effective way to help people lose weight and decrease chronic disease risk. 1 Diet self-monitoring assists with weight management 2 and is considered the cornerstone of behavioral treatment for weight loss. 3 Adherence to self-monitoring 4 and receiving personalized feedback on diet 5 6 are associated with improved weight loss, but diet self-monitoring tends to decline over time. 7 8 Mobile health (mHealth) technologies hold promise as a way to provide individuals with the ability to self-monitor diet and receive feedback wherever they are. Generally, studies requiring participants to self- monitor diets have utilized paper journal methods, 2 which can be time consuming and tedious for participants. 9 Recently, smartphone cameras have made photo- graphing foods a possibility, making just-in-time food recording possible. 10 Recording food through photographs may be one way to reduce the partici- pant burden for recording foods. In one study, which had users record dietary intake via phone cameras, users gave using the camera phone high ratings of satisfaction and almost all preferred the camera method to traditional pen and paper recording methods. 11 Finding ways to provide quick and low-cost feedback to users based on food photographs has been a challenge. One approach to providing feedback on photos of diet is to utilize crowdsourcing, which uses the input of several users to provide feedback and information, 12 such as in the Eatery application (http://www. massivehealth.com/). Users take pictures of their foods with the Eatery app, rate their meals using a sliding scale from fit (healthy) to fat (unhealthy), and are then prompted to rate the photographs of foods and beverages from other users. In addition, users receive peer feedback as an average healthi- ness score for their own foods and beverages. Figure 1 provides a screenshot of the Eatery app interface for rating and feedback. The Eatery application represents a potential use of mHealth technology to reduce the burden of self-monitoring. However, little is known about the validity of peer feedback using a crowdsourcing model. The goal of the study was to assess how closely crowdsourced ratings of foods and bev- erages were related to the ratings of the same pic- tures by trained raters. A secondary goal of the study was to examine if foods and beverages that should be increased in the diet (according to the US Dietary Guidelines) were associated with higher crowdsourced ratings, and if foods and beverages that should be limited in the diet were associated with lower crowdsourced ratings. We hypothesized that crowdsourced ratings of foods would be similar to those of trained raters comparing a basic rating for crowdsource users (scale of fit to fat) to a more complex rating system based on the US Dietary Guidelines. In addition, we hypothesized that foods and beverages that should be increased in the diet (based on the US Dietary Guidelines) would be associated with higher peer user ratings than those foods and beverages that should be decreased. Turner-McGrievy GM, et al . J Am Med Inform Assoc 2014; 0:1–6. doi:10.1136/amiajnl-2014-002636 1 Research and applications