As reported by the Guardian, the iPhone keeps unencrypted location data in a file on the phone, which is also backed up to your computer via iTunes. Some security researchers found the data and have put together a tool for you to extract that data easily. I followed this post to recover the raw location data from my phone, over 11000 records. Here is that data, on a map.
Update: Apple released a Q+A on this data, concluding it is not all your location data but a subset of the location database they keep, cached on the device to help the location services work faster when required.