The abundance of data, coupled with cheap and widely-available computing and storage, has revolutionized science, industry and government. Now, to a large extent, the bottleneck to obtaining actionable insights lies with people. To extract knowledge from data, complex computations that are often out of reach for domain experts who do not have training in computing need to be carried out. Additionally, there is much room for error in the path from data to decisions, from problems with the data and computations to human mistakes. I will present a set of techniques and systems we have developed to guide users in and support the interactivity required for exploratory analyses. I will also reflect on the importance of provenance in this context, not only for transparency and reproducibility purposes, but to enable experts to debug and build trust in the insights they derive.