I totally buy that. It really fits. Angel only seemed to be interested in Buffy (Drusilla, too) when they appeared to be moving forward past him. Then he all of a sudden has time for them. I've always wondered if he would've returned to SD had Buffy not mentioned Riley in Sanctuary.