People often ask whether spending time on social media is good or bad for us. To answer this question, researchers need accurate ways to measure how much time people spend on platforms like Facebook, among other things. The most common approach, found in the vast majority of published studies, is through survey questions asking participants how much time they spent on these platforms. However, participants’ reports of their own use have well-documented limitations. Participants may not report accurately, because they either can’t recall or don’t know. Keeping track of time is hard, and people may report in biased or skewed ways. Some people may be more prone to recall errors. Further, validating these self-report measures is challenging in the absence of data from internal server logs.
Our aim is to provide researchers with validated self-report time measures that more closely capture people’s actual time spent on Facebook. In our latest paper, “How Well Do People Report Time Spent on Facebook? An Evaluation of Established Survey Questions with Recommendations” (CHI 2020), we evaluate common survey questions from the literature, provide recommendations to researchers, and provide translations for 14 languages.
We compared data from 10 self-reported Facebook-use survey measures deployed in 15 countries (N = 49,934) against data from Facebook’s server logs. We found that:
Participants asked “How many hours a day, if any, do you typically spend using Facebook?” overestimated their time on Facebook by an average of 3.2 hours per day.
Informed by these results, we recommend the following to researchers aiming to measure time spent on Facebook:
1. To reduce measurement error, we recommend that researchers ask participants to report data from time management tools like Your Time on Facebook rather than try to estimate it themselves.
2. When time spent must be collected via self-report, we recommend the following wording from Ellison et al. (2007), which had the lowest error in our study.
3. Because self-reports of time spent are imprecise, we suggest that researchers not use these values directly but rather interpret people’s self-reported time spent as a noisy estimate of where they fall on a distribution relative to other respondents.
While our focus here is “time spent” questions, because these are very common in the literature, a growing body of studies shows that merely examining the amount of time an individual uses social media is inadequate for many questions of interest (such as how social media use might be associated with loneliness, social comparison, or academic performance). Instead, we recommend focusing on how people use social media, as discussed in the following studies:
Beyond these implications to researchers, we hope tools such as Your Time on Facebook provide people with more insight into the time they spend on our platform, and foster conversations around their perceptions of use and online habits. Connecting our own perceptions of Facebook use (“How is the time I spend on Facebook good/bad for me?”) and what scientific research tells us about social media’s impact on our lives is also crucial. In that regard, we hope insights from our study provide readers with tools to critically engage with science communication on social media use and well-being.
With the evolution of the platform with new features and shifts in how people use Facebook, even the strongest survey measures are likely to evolve. That said, employing a stable set of established measures is an important methodological practice for researchers to support comparative work within the scientific community. We hope to make a positive contribution by providing such validated measurements that support international, academic, and comparative work on the impact of Facebook on people’s lives.