Rep. Patrick Kennedy had decided not to run for re-election, ending the Kennedy legacy in the Senate and House after almost 60 years.
Kennedy says he wants to spend time building a family instead of building a life in politics.
Kennedy, who is 42, has never married and has never had children.
“It’s pretty simple in this respect: I went through something that caused me a great deal of soul searching and self-reflection,” Kennedy said. “Right now, a personal life is of greater value. Emotional connections that are real and loving and personal just trump everything else.”
He will announce on Sunday night that he has decided not to seek re-election, saying his life is "taking a new direction" since his father died.
The Rhode Island Democrat recorded a message for a Rhode Island television program on Sunday night.
"Now having spent two decades in politics, my life is taking a new direction, and I will not be a candidate for re-election this year," Kennedy says.
Kennedy was in a tough race against a well-financed Republican opponent, but the eight-term congressman was still expected to win. He spoke about his father's death in his remarks for Sunday.
"Illness took the life of my most-cherished mentor and confidante, my ultimate source of spirit and strength," he said, while a black-and-white photo of him as a boy sailing with his father appeared on the screen. "From the countless lives he lifted, to the American promise he helped shape, my father taught me that politics at its very core was about serving others."
Kennedy thanked the voters of his district for supporting him through episodes of alcoholic behavior and mental breakdowns.
"When I made missteps or suffered setbacks, you responded not with contempt, but compassion," he said. "Thank you for all the times you lifted me up, pushed me forward."