I also heard a story once, im sure that it was brought downs somewhere, that there was a shidduch suggested to a girl, and she turned it down because the boy suggested was lame. He couldnt use his legs. It turns out, that before the boy was born, he knew that his zivug wouldnt be able to use her legs and he asked Hashem to give him the nisayon instead of her because he didnt want her to suffer. Theres probably much more to the story, but its making the same point.
Dont get caught up so much in exterior and say no for silly reasons. It is all sheker. Look at the middos of the person, their goals in life, if they respect you and other people-that’s what matters in a marriage. And sometimes that will be enough to find chein in each others eyes.