I thought he was actually pretty tolerant of Spike in S6 until he sees him with Anya and righteous or not I think he would've been just as angry if it had been Riley

Oh, I can't agree with this at all. I could never see Xander taking after Riley with an axe! He would have been incredibly hurt (even though he didn't have a leg to stand on since he was the one that called off the wedding), possibly angry, but not insane. It was the fact that Anya was with Spike that set him off. His tolerance of Spike in S6 was pretty limited, too. He didn't waste any time kicking Spike to the curb in Afterlife. Spike was acceptable when Spike was useful.