"I would've incapacitated you. I think. Depends on how the fight went. I didn't kill Fury when I could've. Natasha I tried to kill, but only because she was giving me an honest to god fight. If he had been there, I know what he would've had me do. To see if I would."

It definitely features in some of his dreams. Some of his waking thoughts, too. Maybe he wouldn't hurt Coulson knowingly, but what if he gets caught off guard, gets distracted, what if everyone is wrong and there's still some undetectable part of him that's got a switch just waiting to be flipped...