Napoleon was exiled from France two times: first to Elba in 1814 and later to Saint Helena in 1815.
His first exile followed the collapse of French resistance after the disastrous invasion of Russia, the allied occupation of Paris, and his abdication in April 1814. The victorious allies allowed him to retain the title of emperor while sending him to Elba, a small Mediterranean island. He escaped in 1815 and returned to France, beginning the period known as the Hundred Days.
After his defeat at Waterloo and second abdication in June 1815, Napoleon surrendered to the British. He hoped to live in America, but Britain instead sent him to the remote South Atlantic island of Saint Helena. He remained there under close supervision until his death in 1821.
The main confusion is between exile and military defeat. Napoleon suffered several defeats, but only the post-1814 and post-1815 settlements produced the two formal exiles counted in this question.