The Hammers were left shell-shocked at the break as they found themselves 3-1 behind.
Jordon Mutch struck first with an impressive solo goal for Blues, bamboozling Abdoulaye Faye with some trickery before firing a shot past an unsighted Rob Green.
And Blues doubled their advantage moments later when Marlon King beat the offside trap before keeping his cool to slot home.
The home supporters were beginning to voice their displeasure before Vaz Te headed in his fourth goal in as many games in first half stoppage time.
However, Blues went straight up the other end and restored their two-goal advantage when Chris Burke fired a volley into the ground which somehow bounced into Green's net.
Carlton Cole pulled one goal back before Vaz Te lashed in the equaliser from the spot which was awarded after Henri Lansbury's shot hit Burke's arm.
West Ham's recent home form has threatened to scupper their bid to immediately return to the Barclays Premier League and the Blues broke the deadlock in the 27th minute thanks to a terrific effort from Mutch.
The midfielder won the ball with a sliding challenge before launching a counter attack following a one-two with King.
Mutch drove straight at Faye, creating space with a step over, before squeezing a left foot shot into the bottom corner.
Chris Hughton's side were then two goals to the good just three minutes later courtesy of King.
Cole was robbed in the middle of the park and King, played onside by Winston Reid, collected Keith Fahey's pass before opening his body and placing the ball past Green.
With murmurs of discontent circling around the ground, Vaz Te's goal offered some hope. Mark Noble's corner was hung up to James Tomkins and he headed the ball across goal for Vaz Te to nod in.
But Blues had their two goal advantage back just over a minute later when Burke hit a volley into the ground which somehow looped over Cole and into the net.
Allardyce threw Lansbury on at half-time and he nearly had an immediate impact after Peter Ramage clumsily clattered into him in the penalty area, although referee Jon Moss ignored the Arsenal loanee's appeals.
The hosts were incensed when Cole was also denied a penalty after Steven Caldwell pulled him back, but the former England striker was on target in the 70th minute.
Kevin Nolan lifted the ball back to Cole from the byline and he controlled the ball before smashing a left-foot half-volley past Boaz Myhill.
The home side flooded forward in search of an equaliser with Cole and Danny Collins firing narrowly over.
And referee Moss awarded a spot-kick to the hosts right at the death when Lansbury's vicious shot struck the raised arm of Burke, who was in close proximity to the effort.
Vaz Te stepped up and crashed his penalty straight down the middle, into the roof of the net, to earn the Hammers a share of the spoils.
Source: DSG