If you leave the present to see what your future was like you'll find out that you disappeared from the timeline the moment you started traveling, and the world went on without you.
Which is how the predecessor culture found itself mostly wiped out. They discovered time travel, went to both ends of the universe, decided heat death sucked, waged a war on the laws of thermodynamics, won, and went to a place where their culture could live forever, that being outside of the normal timeline. They didn't stop to think about who might end up ruling the world they left behind, it turned out that a world without them in it would result in a very strong, very populous, very warlike people becoming the next civilization to learn how to step outside of time.
All of the groundwork already laid (hey, there's air and energy here) they were able to quickly start to do what they did best: wipe out the local population. The predecessors had better technology, but they lacked numbers. The war wiped out most of both sides (and changed the main timeline beyond recognition*), leaving the space ripe for the current breeds of time travelers to settle.
I haven't quite figured out what happens if you insert yourself back into the main timeline so that you can talk to yourself before you left it, which is somewhat problematic because that's where causality can really get confusing. The simplest solution leads to some truly annoying questions.
*The predecessors were from the fourth great Carthaginian empire. Hannibal won in the original timeline.