Blues started the game well and took the lead through Seb Larrson on 26 minutes at The Walkers Stadium on Tuesday night.
However, the lead was rather short-lived as the hosts levelled just six minutes later through Matty Fryatt.
Substitute Kevin Phiilips gave Blues a 2-1 lead when he latched onto a Larsson corner on 78 minutes.
The Foxes once again equalised through Levi Porter with just two minutes of normal time remaining to make the score 2-2.
With the game drawing to a close and a draw looking the most likely result, Phillips sealed a 3-2 win for Blues with his second goal of the game just seconds from the final whistle.
Blues: Doyle, Parnaby, Martin Taylor (Jaidi h/t), Ridgewell, Murphy, Larsson, Nafti, Carsley (Mutch 86), McSheffrey (Jerome 67), Bent (Phillips 67), McFadden (O'Connor 67).