While in any post or page in EDIT mode, click on desired image. It will "gray out" and you will see two buttons appear at the top left.
Click the first button to edit the image.
The "Edit Image" window will pop up. Enter desired text in the caption field.
Click the update button. The "Edit Image" window will close.
Don't forget to save the page or post to make your update permanent.