Shirogane takes that to mean that she would never visit. And he wouldn't ask her to visit, because he knows if she promised, it would only cause problems for her.

At least, he knows that he could never make her put down her job as an Undine for something as trivial as visiting him on Man-home.

He doesn't look at her as he gives her a noncommittal, "Probably."