Voting Advice Applications (VAAs), which provide citizens with information on the party that best represents their political preferences, are often cited as evidence of the empowering capabilities of digital tools. Aside from the informational benefits of these voter guides, observational studies have suggested a strong effect on political participation and vote choice. However, existing impact evaluations have been limited by a reliance on convenience samples, lack of random assignment, or both. This raises questions about self-selection and the precise mechanisms underlying how voters learn about politics. Here, we provide evidence from a field experiment with survey outcomes conducted with a sample of over 1,000 German citizens in the 2017 federal election campaign. Using linked panel survey and digital trace data combined with a randomized encouragement to complete a VAA, we are able to assess respondents’ compliance with treatment as well as to observe how the use of this tool affects political behavior, attitudes, media consumption, political knowledge, and even social media activity. Our findings reveal that the overwhelming consensus in favor of positive effects on turnout and vote choice should be treated with caution, as we find no such effects. Rather, we show that the actual virtue of VAAs in a complex online information environment is in increasing knowledge about parties’ positions on issues - exactly the kind of information these tools were designed to provide.