Lessons Learned in Building a Modern Microscopy Data Infrastructure at NIST

In the past decade, the electron microscopy (EM) community has witnessed a paradigm shift in nearly every aspect of the experimental workflow. Hardware advancements and innovative data collection methods continually push the limits of performance in imaging and spectroscopy [1], while novel analysis techniques adopted from the data science community have led to the emergence of “datadriven” microscopy [2], where previously hidden insight arises from statistical analyses of large amounts of data. These advancements have given rise to new challenges in how we store, manage, and process the ever-increasing amounts of data produced, requiring the adaptation of EM researchers and collaboration with data scientists to extract the most insight.