Digital literacy is receiving increased scholarly attention as a potential explanatory factor in the spread of online misinformation. As a concept, however, it remains surprisingly elusive, with little consensus on definitions or measures. Building on work in communication studies and sociology, we provide a unified framework of digital literacy for political scientists and introduce survey items to measure it. Using a novel purposive sampling approach, we then validate our measure against real-world benchmarks of ground truth. There exists substantial variation in levels of digital literacy in the population, which we also document is correlated with age and could confound observed relationships. However, this is obscured by researchers' reliance on online convenience samples that select for people with computer and internet skills. We discuss the implications of this sample selection bias for effect heterogeneity in studies of online media effects on political behavior.