By way of the venous system or direct penetration, particles such as thrombi, bullet fragments, and shrapnel can become trapped in the heart and disrupt cardiac function. The severity of disruption can range from asymptomatic to fatal. Injuries of this nature are common in both civilian and military populations. For symptomatic cases, the conventional approach is removal of the foreign body through open heart surgery, which comes with high perioperative risks and a long recovery period. To circumvent these disadvantages, we propose a minimally invasive surgical approach for retrieving foreign bodies from a beating heart. This paper describes the first use of 3D transesophageal echocardiography (TEE) for steering a robot. Experiments demonstrate the feasibility of using 3D ultrasound to both guide and track a robot as it pursues a foreign body, with an RMS error of 1.6 mm in a laboratory setup. Results also support the hypothesis that direct pursuit of the foreign body may exceed the capabilities of conventional surgical robots, necessitating alternate retrieval strategies.