Notre Dame tumbled to No. 4 in the BCS standing after their narrow victory against Pitt.
Oregon, who defeated USC, jumped to No. 3.
It now seems the Irish will need two of the three teams ahead of them to lose a game to allow them into the national championship picture. Alabama almost lost to LSU on Saturday but pulled out a last second victory.
Kelly denied he was looking at the BCS rankings.
"Maybe I should say the politically correct, yes, you know. We really hope -- I don't really care about that," Kelly said after Notre Dame's 29-26 triple-overtime win on Saturday over Pitt. "Everybody says I should care. I care about my football team. I care about getting better. We're really just focused on ourselves. We can't worry about all those other things. We really can't.
"I could be politically correct and say, 'Yeah, I care, we're going to watch it.' But to be honest with you, we've got a lot of work to do with our football team."