|Swift and Styles hold hands while leaving|
The couple teamed up to sing a rendition of Kenny Rogers and Dolly Parton's classic 'Islands in the Stream' at the Hudson Hotel after the 18-year-old hunk and his bandmates had performed to 20,000 screaming fans at Madison Square Garden in New York.
A source told The Sun newspaper: "It was brilliant. There was a huge cheer when they got up."
Following the performance, Harry wrote on Twitter: "Last night was the best thing I have ever done.
"We're on an Island over the Stream... (sic)"
Taylor, 22, and Ed Sheeran, 21, then joined Harry and his fellow One Direction bandmates - Niall Horan, Zayn Malik, Louis Tomlinson and Liam Payne - to sing Backstreet Boys' 1998 hit 'I Want It That Way', and the flame-haired singer teamed up with the 'Little Things' hitmakers for Vanilla Ice's 'Ice Ice Baby'.
Harry and Taylor - who went for a romantic stroll around New York's Central Park prior to the gig - also performed Sir Elton John and Kiki Dee's classic hit 'Don't Go Breaking My Heart' before heading back to their hotel in the early hours.
One partygoer told the Daily Mirror newspaper: "It was the perfect song for a new couple - they were both giggling as they sang it to one another.
"It was very fun and very relaxed. Everybody was in really high spirits. You could tell they are very comfortable in each other's company.
"They were still smiling when they disappeared back to her hotel hand-in-hand in the early hours of the morning."
Zayn Malik was accompanied at the gig by his girlfriend, Little Mix singer Perrie Edwards, and her brother Jonnie also teamed up with One Direction - whose rider featured a host of calorific treats including Snickers, KitKats and cupcakes - to sing at the afterparty.