Algeria came back from a goal down to eventually earn a comfortable win in their opening game.
After dominating the first half, South Africa took a 51st-minute lead when Thuso Phala finished a flowing move.
Tokelo Rantie missed the chance to double the lead when his penalty hit the bar and Thulani Hlatshwayo headed past his own keeper to level the score.
Faouzi Ghoulam put Algeria ahead with a fierce shot and Islam Slimani squeezed the third under goalkeeper Darren Keet.
South Africa will feel they missed the chance to cause a shock – especially after having the better of things for two-thirds of the match.
It was only Algeria goalkeeper Rais M’Bolhi who kept them at bay in a largely disappointing first half.
In the 23rd minute he tipped a 35-yard shot from Doncaster midfielder and South Africa captain Dean Furman on to the bar.
Then in the 42nd minute he bravely dived at the feet of Sibusiso Vilakazi as he was going round him and 60 seconds later was called into action again to save Andile Jali’s shot with his knees.
South Africa went ahead early in the second half when Phala slotted the ball home from eight yards after lovely build-up play from Rantie and Vilakazi.
Three minutes later Vilakazi was brought down in the penalty area by Aissa Mandi – but Bournemouth striker Rantie hammered the spot-kick against the top of the bar and that proved to be the turning point.
Algeria levelled when Yacine Brahimi slung in a cross from the left and Bafana defender Hlatshwayo managed to loop a header over Keet and into his own net.
Four minutes later Algeria were ahead when left-back Ghoulam powered his way into the left-hand side of the area and hammered his first international goal past Keet.
Slimani made the match safe in the 82nd minute when his shot from the edge of the area trickled through Keet and rolled over the line.
The victory puts Algeria top of Group C on goal difference ahead of Senegal who beat Ghana 2-1.
