Timothy Olyphant has been romantically linked with only one woman, his wife Alexis Knief, since his college days.