Congratulations to you both!!!
I can practically FEEL you stressing out over this! Poor girl! And I know your attention is all for the baby and the trauma you're anticipating about leaving him, etc., etc., etc.
I'd suggest that, without putting aside your concern for your child, you put at least as much thinking time into your relationship with your husband. In a monagamous society, the only acceptable person for him to create romance with is you, and vice versa. So if you don't create that with and for each other, life just loses so much of its spark and sizzle.
Yes, you've gone through lots of changes in becoming a mom. And there will be lots more to go through... first tooth, first day of school, first girlfriend, first car... and on and on. How much better if you have a close and loving partner to share all that with.
So take this time to cultivate that and re-energize the wonderful and loving relationship that created that gorgeous child!
Best of luck!