Human memory is a lot trickier thing than we think. It's not about accessing a video recording. It's more like playing a game of Operator.
Embedded Link
Why Can’t You Remember Being a Baby? – Mental Floss
Why Can’t You Remember Being a Baby?
Google+: View post on Google+

I always assumed it was like looking for something in the very back and bottom of a packed garage. 😛
Using that analogy, the something is also packed in plain, undistinguishable brown cardboard boxes, and doesn't even look like what you think it might when you open it up.