Pep Guardiola's team were in control for long periods at St Mary's, but it wasn't until the 55th minute that City captain Kompany put them ahead with his first goal since August 2015.
Leroy Sane and Sergio Aguero made sure of the three points with further goals in the 77th and 80th minutes respectively.
With Manchester United beginning to breathe down their necks in the pursuit of a top-four place and with games in hand, City needed this result, only their second league win in six matches, to move one point above Liverpool.
It was an especially cathartic moment for Kompany, who was making only his fourth league start in yet another injury-plagued campaign for the Belgium defender.
Aguero might have scored twice in the first four minutes.
First he escaped his marker at the far post but could only volley Gael Clichy's cross wide.
Then he made himself a yard of space on the right of the penalty area but pulled his shot across goal.
Neither miss made Aguero more reluctant to shoot. From a corner-kick on the right, he tried an 18-yard volley, which went about the same distance wide.
For 15 minutes Southampton were barely a factor in the game as City passed the ball round and through them.
But then they produced an incisive move that should have brought them the opening goal.
Nathan Redmond's pass between Jesus Navas and Kompany freed Manolo Gabbiadini on the left, and he rolled the ball back into the path of Dusan Tadic 15 yards out.
However, the Serbia playmaker's first-time shot sent the ball well over the crossbar.
That, though, was an isolated incident as Southampton spent most of the rest of the first half chasing shadows.
In the 36th minute, Aguero's attempted cross looped up off Cedric Soares and David Silva, running in, seemed certain to score, only to half-volley into the side-netting.
Clinical execution
The next error seemed to be made by referee Neil Swarbrick when he awarded a corner after Southampton goalkeeper Fraser Forster rushed out of goal as Sane charged through.
Swarbrick ruled that a swipe of Forster's right glove had made contact with the ball, but replays showed little evidence of that and suggested that he had hit Sane's foot and that a penalty should have been awarded.
If Southampton had a hope at that point, it was probably that Claudio Bravo, preferred in City's goal to Willy Caballero, would make an error, and after he could only flap at a cross from the left, James Ward-Prowse returned the ball to the near post, where Steven Davis got his head to it only to nod it too high.
Ten minutes into the second half, City finally made the breakthrough.
It might have come from Navas's rising shot from 25 yards that forced Forster to touch the ball over the bar, but instead it came from the resulting corner.
Gabbiadini missed Silva's kick from the left and Kompany, making only his third appearance of 2017, muscled his way past Ryan Bertrand and headed past Forster from six yards before celebrating wildly.
City doubled the lead in the 77th minute with a goal of devastating simplicity but clinical execution.
Kevin De Bruyne exchanged passes with Silva just inside his own half and ran clear before sliding the ball to his left for Sane to finish.
Three minutes later and the scoreline more fairly reflected City's dominance.
De Bruyne collected Navas' pass on the right and chipped the ball across for Aguero to head in.