We could never return back in a straight line. In terms of explanation, if you go straight on a flat universe there's a "point" where you would reach the end, which would make you to turn back and change the path making therefore a small curvature in your path. "Unless the Universe repeated on a scale that was less than about 15 billion light years in diameter today, we'd have no way to ever return to our original starting point by traveling in a straight line"