You need not feel guilty about missing anyone. As missing someone who is emotionally significant to us is not bad. As such you are married and cannot afford to be in an emotional relationship, outside your marriage. And if he is going away from you it is a reasonable thing to do under this situation so let the time pass and you grow out of this.