One person was arrested for possessing drugs, another for breaching bail and a third for carrying weapons - large fishing sinkers - Victoria Police's Superintendent Tony Silva said.

Officers also detained several people who were then released to try and prevent them causing trouble at the tightly-controlled rallies on the foreshore.

Hundreds of police took to the air, sea and land to control the event, including the dog, mounted and riot squads, from Saturday morning until 4.30pm.

"I certainly felt we had it under control," Supt Silva told reporters.

Several hundred people came for the rallies, the first held at 12pm by anti-racism campaigners ahead of a right-wing event at 1pm organised by convicted criminals Blair Cottrell and Neil Erikson.

Queensland Senator Fraser Anning, who used the Nazi-associated phrase "final solution" in his maiden speech, stood with right-wing extremist Cottrell at the rally.

Senator Anning, who now sits as an independent after being booted from the Katter Australia Party following his defection from Pauline Hanson's One Nation, uploaded several videos to his Facebook page from the rally.

In the videos, Senator Anning stands with Cottrell, poses for photos and makes inflammatory remarks about migration.

The far-right group chanted "Aussie, Aussie, Aussie, oi, oi, oi" while the counter protesters yelled "Migrants welcome, racists are not".

After speeches from both sides, the opposing groups dispersed onto nearby streets and to the front of Luna Park, with minor scuffles breaking out.

At least one person was pepper-sprayed.

Anti-racist activists drove past the event with loud speakers on the tray of a ute, shouting slogans, but when it stopped because of the crowd it was set upon by the opposition.

Cottrell and Erikson in 2017 were convicted and fined for inciting contempt and ridicule of Muslims by making a video in which they beheaded a dummy with a toy sword in a protest against the building of the Bendigo mosque.

Cottrell is appealing.

The pair claim the rally is a response to recent incidents in which youths have mugged people along the bay.

Erikson last week confronted a group of young men of African background who were playing soccer at St Kilda, prompting police intervention.

St Kilda and nearby Caulfield, areas with high Jewish populations, have also experienced a blitz of anti-semitic vandalism.

The Emmy Monash Jewish aged-care centre in Caulfield was plastered with a swastika by neo-Nazi group Antipodean Resistance and a theatre graffitied in recent days.

Race Discrimination Commissioner Chin Tan said in a statement he condemned in the strongest possible terms the apparent racist and race-based motivation behind the far-right rally.

"All Australians have a right to protest peacefully ... It is, however, never acceptable for people to act or conduct themselves in ways that are likely to, or intended to, inflame and incite racial fear and tensions, hatred and violence."

Australian Associated Press