What happens if we never settle space?

“Never” is a really long time. So, for the sake of discussion, let’s define it as a human lifetime plus some margin—say, within this century. What if humanity has not built a space settlement anywhere in the solar system—not the Moon, Mars, or free space—within that time? Does that matter? Should it matter to any of the people who are currently interested in space settlement? Should they, will they, abandon their interest and enthusiasm?