Aliens or time travelers?

So, there are all these theories out there about how ancient cultures accomplished so much with such simple technology without ever having contact with one another (think Egyptian and Incan temples). One rapidly developing theory is that an alien race visited these ancient cultures and provided them with the tools/technology necessary. This theory has risen due to the fact that the structures these cultures erected could not have been possible for them to create on their own with their simple tools, but rather needed more sophisticated machinery that would be like slightly more advanced versions of modern day equipment. So my question is this: rather than 'ancient aliens' having visited these cultures, what if it was merely a group of humans that had perfected the science of time travel and gone back in time? With the seemingly limitless possibilities in modern science, wouldn't that be possible? I feel like our descendants would have more desire to interact with their ancestors than would a race of alien beings that have no connection to us whatsoever. What's your theory?