Something about your new manager's behavior does not compute. Your calculator, no matter how basic, still had the capacity to do what he wanted. Surely this manager must have something better to do than wander up and down the halls making negative comments about a calculator.
You should not go on the offensive or the defensive on this matter. You should sit on the sidelines. This whole incident merits nothing more than a shrug of the shoulders, and that's just about all you should do if someone asks you about it.
Your manager may have been looking for a way to demean you, and his motivations could include dissatisfaction with your performance, personal insecurity regarding your performance, or a misguided sense of competition. None of this makes him look particularly effective in any sense of the word.
The real issue is how he will react when he faces a larger issue. He is your new manager, but one has to wonder if he is new to management, too.
| 
