“Photographs open doors into the past, but they also allow a look into the future.”—Sally Mann

“In photography there are no shadows that cannot be illuminated.”—August Sander

When you’re feeling sad, the people around you probably know it. Moody playlists, slumped shoulders, drawn-out sighs—there are many ways we signal to the rest of the world when we’re having a down day. It’s not all that much of a stretch, then, to imagine your Instagram posts might look happier when you’re feeling happy, and sadder when you’re feeling sad.

What if you were feeling depressed, but didn’t quite know it yet—would your depression still show up somehow in the photos you shared online? This possibility got us thinking: how might we combine what psychologists know about depression, with what data scientists know about analytics, to develop a quantifiable approach for evaluating mental health on Instagram? The results of our work suggest that early-warning signs of emerging mental health issues like depression can be observed in Instagram posts, even before any clinical diagnosis is made.

We asked people to share their Instagram posting histories with us, along with details about their mental health history. By design, roughly half of our study participants reported having been clinically diagnosed with depression sometime in the last three years. All-in-all, we collected 43,950 photos posted to Instagram for analysis.

Using findings from clinical psychology research, we identified several visual and behavioral markers associated with depression that seemed like good candidates for measurement. For example, individuals suffering from depression exhibit different preferences for color, shading, and brightness in imagery, compared to healthy individuals. Pixel analysis of the photos in our dataset revealed that depressed individuals in our sample tended to post photos that were, on average, bluer, darker, and grayer than those posted by healthy individuals.

Right photograph has higher Hue (bluer), lower Saturation (grayer), and lower Brightness (darker) than left photograph. Instagram photos posted by depressed individuals had HSV values shifted towards those in the right photograph, compared with photos posted by healthy individuals.

Depression is also characterized by reduced or avoidant social engagement. Social engagement involves other people, so we speculated that one rough measure of sociability might be the average number of people that show up in the photos you post. We wrote a face detection algorithm to count the number of faces that appeared in each posted photograph. It turned out that depressed individuals posted significantly fewer faces per photograph, compared to healthy individuals.

Even the way depressed and healthy people chose to present their photos on Instagram was different. Instagram offers a series of ready-made filters that adjust a photo’s appearance. Among healthy users, we observed that the most popular filter was Valencia, which gives photos a warmer, brighter feel. Among depressed users, however, the most popular filter was Inkwell, turning it black-and-white. In other words, people suffering from depression were more likely to favor a filter that literally drained all the color out of the images they wanted to share.

Blue bars indicate disproportionate use of a filter by depressed compared to healthy participants, orange bars indicate the reverse. Gosling did not participate in the study.

We were able to observe these differences reliably, even when only looking at depressed users’ posts made prior to receiving a clinical diagnosis of depression. These and other recent findings (1, 2, 3) indicate that social media data may be a valuable resource for developing efficient, low-cost, and accurate predictive mental health screening methods.

We do feel strongly that there’s an important ethical discussion that must occur in step with these technological developments, regarding data privacy and the implications of applying sophisticated analytical tools in an online medium which doesn’t forget. Even so, the possibility that social media analytics may offer a means of getting help faster to people in need is important, and should be explored further.

You can read additional details on our study here, where it was published by EPJ Data Science today.