A method to reconstruct motion from sequences of tagged magnetic resonance (MR) images is presented. MR tagging is used to create a spatial pattern of varying magnetization so that objects which may otherwise have constant intensity are textured; this reduces the motion ambiguity associated with the aperture problem in computer vision. To account for the decay of the tag pattern, a new optical flow algorithm is developed and implemented. Velocity fields estimated using this algorithm are used to recursively update the implied motion reference map over time, thereby tracking the motion of individual particles. If a segmentation of the object is known at the time the tag pattern is created, then an object may be selectively tracked, using the estimated reference map to update the object's position as time progresses. Results are shown for actual MR phantom data.