It is really frustrating to live the way you live right now.
Sadly your story offered no glimmer of hope.
There is no way one person can be making all the effort to keep a marriage. Frankly, i'm not about him having remorse or not. I am concerned, he recognizes the presence of a problem but somehow refuse to see himself as part of it or humble himself to seek a solution with you.
That means he is not ready to change or make any adjustment.
Taking care of the family, nice inlaws, probably zero cheating might not be an issue especially for ultra settling and enabling wives but your emotional health is really at stake.
This relocation you talk about, is it back to Nigeria?
Coz if he is messing up in Canada like this, expect a proper demon when he comes back here. Even the nice inlaws you will lose by the time he orchestrates bad wife scenarios and calls meeting and, takes advantage of the sickening overwhelming male priviledge here. That question he posed to you is a sneak peek/expo of his grand plans soon for you. Say yes by staying back and you lose your chance. Take a break, get settled, build yourself, live and enjoy your life for you. Then maybe his brain and ego may reset or you both move to some other people or most importantly, you get to live for real and have your son have a complete strong mom who will show motherly love not spousal one and groom him to be a real man.
May God help you.