Obama and his wife, Michelle, boarded the plane under cold, rainy skies for the long flight. Bush and his wife, Laura, and Clinton, already were on board.
Former President Bill Clinton was traveling separately from Rio de Janeiro, where he was attending a Clinton Global Initiative event. And former President Jimmy Carter also planned to join the group in Johannesburg.
George H.W. Bush is the only living president who will not attend. His spokesman said the 89-year-old no longer is able to travel long distances.
The American leaders will join dozens of other dignitaries and tens of thousands of mourners at the memorial service at a Johannesburg stadium. Mandela will be buried Dec. 15, after a state funeral in his hometown of Qunu.
Obama is to be one of the speakers at Tuesday's memorial. White House officials said Obama was hoping to meet with Mandela's widow, Graca Machel, and pay respects to the extended Mandela family. Also traveling with Obama on Air Force One were Attorney General Eric Holder and Susan Rice, Obama's national security adviser.
About two hours after Obama's departure, Vice President Joe Biden, who stayed behind, visited the South African embassy in Washington with his wife, Jill, to sign a condolence book in Mandela's memory. "Through his unflagging, unflinching commitment to human dignity and his willingness to forgive, he inspired us and challenged us all to do better," Biden wrote.
Obama has called Mandela a personal hero, and his own political rise has drawn inevitable comparisons to the former South African leader. Each was his nation's first black president and a winner of the Nobel Peace Prize — though those close to the U.S. leader say he is aware of the vast differences in the politicians' experiences.
"President Obama would believe that the challenges he has faced pale in comparison to those faced by President Mandela," Valerie Jarrett, Obama's senior adviser, said in an interview this summer before the president made a trip to South Africa.
Mandela was hospitalized during Obama's visit, preventing a meeting between the two men. Obama did, however, meet with several members of Mandela's family and made an emotional visit to Robben Island, standing with his wife and two daughters in the tiny cell where the South African leader spent 18 of his 27 years in prison.
Obama and Mandela met in person only once, a hastily arranged 2005 meeting in a Washington hotel room when Obama was a U.S. senator. A photo of the meeting hangs in Obama's personal office at the White House, showing a smiling Mandela sitting on a chair, his legs outstretched, as the young senator reaches down to shake his hand. A copy of the photo also hangs in Mandela's office in Johannesburg.
The two men did speak occasionally by telephone, including after the 2008 election, when Mandela called Obama to congratulate him on his victory. The U.S. president called Mandela in 2010 after the South African leader's young granddaughter was killed in a car accident. Obama also wrote the introduction to Mandela's memoir, "Conversations with Myself."
Mandela had already shaped Obama's political beliefs well before their first encounter. As a student at Occidental College in Los Angeles, Obama joined protests against the school's investments during South Africa's apartheid era. In 1981, Obama focused his first public political speech on the topic.
"The day that he was released from prison gave me a sense of what human beings can do when they're guided by their hopes and not by their fears," Obama said in remarks shortly after Mandela died. "And like so many around the globe, I cannot fully imagine my own life without the example that Nelson Mandela set, and so long as I live I will do what I can to learn from him."
Follow Julie Pace on Twitter: http://twitter.com/jpaceDC
Copyright 2013 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.