It was just the start to his new career with the Premier League giants Arsenal that Australia goalkeeper Mat Ryan, who had moved to the club on loan from Brighton 24 hours earlier, would have been desperate to avoid.

Ryan, brought in as an understudy to Bernd Leno for the rest of the season, watched from the substitutes' bench as the German was beaten in the 24th minute when Gabriel inadvertently turned in Kyle Walker-Peters' cross.

The defeat of the holders and 14-time winners was the big story of Saturday's action although Manchester City had to survive a scare, facing elimination at fourth-tier Cheltenham Town as they were one down with 10 minutes left before ending up 3-1 winners.

The other big guns also went through to the last-16, with West Ham, who beat Doncaster Rovers 4-0, next set to play the winners of either Manchester United or Liverpool, who play the big tie of the round on Sunday.

Fellow Premier League sides Sheffield United and Brighton are also through after triumphing at home.

United, bottom of the English top-flight, dispatched third-tier League One side Plymouth Argyle 2-1 while Brighton struggled before overcoming Blackpool 2-1.

Bristol City, of the second-tier Championship, will be Sheffield United's next opponents following a 3-0 success at Millwall, while Brighton await the winner of Brentford against Leicester City on Sunday.

Another Championship side Barnsley advanced with a 1-0 victory against Norwich City to set up a meeting with either Chelsea or Luton Town in the next phase.

Earlier, all eyes had been on St Mary's as Arsenal's reign ended, with manager Mikel Arteta calling it the "end of a dream".

"We're really sad to be out of the competition," said the Spaniard, who had made seven changes to his side and was without captain Pierre-Emerick Aubameyang, who withdrew for personal reasons in the hours before kick-off.

It was Southampton's first-ever victory over Arsenal in the FA Cup after five attempts and it was hard to disagree with Southampton boss Ralph Hasenhuttl's assessment when he said the Saints had "more punch and more power."

The Saints now advance to the fifth round, where they will travel to Wolverhampton Wanderers, who knocked out sixth-tier side Chorley to advance on Friday.

There was plenty of drama as League Two side Cheltenham, 72 places behind Manchester City in the league pyramid, went ahead at a sadly empty Jonny-Rocks Stadium with Alfie May's 59th-minute goal.

With less than 10 minutes left on the clock they were still dreaming of one of the great Cup upsets but reality dawned as late goals by Phil Foden, Gabriel Jesus and Ferran Torres spared Manchester City any humiliation.

It was City's 10th successive win in all competitions but Cheltenham depart the competition full of pride after pushing City's slickers to the limit.

West Ham had earlier cruised to victory over third-tier Doncaster with first-half goals from Pablo Fornals and Andriy Yarmolenko before a second-half own goal from Andy Butler and a debut goal from striker Oladapo Afolayan sealed the result.

Australian Associated Press