Weston with late winner (Will T Photography)
The Peacocks were certainly made to scrap for victory, despite taking a 24th minute lead when Jared Lewington was allowed time to bring the ball down before firing past Jaimie Cogman.
And it was no surprise when the Martyrs equalised 10 minutes later when Eliot Richards got on the end of a Kieran Evans cross to slot beyond Taunton `keeper Will Buse.
The second half saw opportunities missed at either end and, thanks to some decent saves from Cogman, a draw looked on the cards.
But in the 7th minute of stoppage time, Ross Stearn broke Merthyr hearts and lifted the roof off the stand, with Taunton`s crucial last-gasp winner.
Further up the M5, sixth-placed Weston-super-Mare moved to within three points of fifth-placed Chesham United, with two games in hand, after beating Swindon Supermarine 1-0 at the Optima Stadium.
And like at Taunton, this game also looked like heading for a draw until there was a late twist.
Weston bossed the possession stats at half-time but the visitors always looked threatening on the break.
However, the Seagulls were dealt a massive blow 8 minutes into the second half when they had Josh Thomas dismissed with a straight red card for a challenge on Jamie Edge deemed reckless by the officials.
But the ten-men re-grouped and more than held their own.
And then in the 89th minute, Weston were awarded a free-kick just outside the area which Dayle Grubb curled past Martin Horsell to win it for the hosts.
Yate Town shrugged off a two-game losing run to inflict a 5-0 drubbing on bottom side Wimborne Town on their own New Cuthbury home.
The Bluebells took a 10th-minute lead when James Harding reacted quickest to a loose ball in the box.
And 5 minutes later, Harding bagged his second when he managed to lob Magpies` keeper Ged Benfield.
A third goal for the visitors arrived just past the hour-mark and it was Harding again, who completed his hat-trick with another neat finish.
On 81 minutes, Harding turned provider for Ollie Mehew to beat Benfield and then 3 minutes later, Luke Hopper made it a nap hand with a 25-yard drive.
The win lifts Yate back to seventh and just two points behind Weston with a game in hand on them.
The evening`s other game saw Tiverton Town leap-frog Salisbury into 15th spot after recording a 3-0 win at the Raymond McEnhill Stadium.
It`s one win in seven now for Steve Claridge`s side, whilst this was a second successive win for the Yellows.
Dan Hayfield scored the only goal of the first half in the 24th minute.
Austen Booth virtually made the points safe with Tivvy`s second on 63 minutes and 3 minutes from time, Jordan Lam added a third for good measure.
Most clubs are looking for volunteers. Find out more on the button below:
www.PitchingInVolunteers.co.ukAll the news and results in one place.
REGISTER