What It Takes to Achieve Precision of a Micron or Less Posted in: Updates
Last Updated: May 12, 2025
We at Acuity sell sensors that can measure to amazing precision. But even so, when customers call us looking for precision on the order of a micron or less, I try to temper expectations. Acuity sensors can be that precise, but the effort required to get that precision in your measurements can be monumental.
Mounting, temperature control, isolation from vibration, etc… There are many sources of outside error that can throw your measurements off, but instead of trying to imagine a possible setup you might need for your application, I’d first like to recommend a video about a machine designed to give true 0.1 micron precision at the NIST labs. The lengths they go to ensure their level of precision is astounding.
We’re sharing the video starting at 4:35, where NIST (the National Institute of Standards and Technology) goes to amazing lengths to isolate their metrological equipment. The video is absolutely worth watching from the beginning, but we wanted to focus directly on what it takes at NIST to achieve measurement precision.
Machine Thinking is a wonderful channel about the history of precision, and I highly recommend it.
It is very important, whatever your measurement application, to analyze and reduce your sources of error. The more precise you need your measurements, the more diligent that analysis needs to be. At Acuity we want more than your purchase, we want your success. I hope this helps provide perspective that helps you achieve it.
Sarah has been our technical support and sales engineer for 5+ years. If you've ever reached out to Acuity Laser for tech support, more than likely, Sarah is the one who helped you.