Half of his closet being flannel doesn't sound too bad, actually. It would be more evidence of Clint settling into their joint life together.

"You're going to make me late one morning and I'll walk into a briefing in a suit and flannel." Right now, he goes through the usual routine he has for going to bed at Clint's place. He doesn't expect anything sexual at all given how gun shy Clint seems right now.

He is content with simply being together again.