Andre-Pierre Gignac scored twice for the visitors, who led 3-1, but Christopher Maboulou's own brace earnt a point for the Corsican hosts.
Maboulou gave Bastia the lead on nine minutes with a spectacular volley but Marseille goalkeeper Steve Mandanda seemed to let the ball go through his hands as he tried to punch it clear.
Gignac's near-post header brought Marseille level before a comical own goal from Romaric put the hosts in front before half-time.
Gignac extended the lead from the spot on 62 minutes after Sebastien Squillaci was penalised for handball.
But just three minutes later, Junior Tallo reduced the deficit with a penalty at the other end after Benjamin Mendy was harshly adjudged to have brought down Maboulou in the area.
Maboulou had the last word, though, with a close-range shot from an angle that slipped inside the far post to ruin Marcelo Bielsa's debut on the Marseille bench.
"It's our first success, in a way, to come back from 3-1 down," said Makelele.
"I congratulate the coach of Marseille as it's a team which is going to cause problems for a lot of teams. We could have lost this match, but I am not concerned as I am still getting to know my players."
The aftermath of the match was however far from celebratory with forty-four French riot and para-military police officers left injured follwoing clashes between rival fans.
In a statement the interior ministry said 10 members of the CRS riot police and 34 gendarmes were "injured or bruised" when violence broke out at the stadium on the island of Corsica.
A source close to the case said scores of supporters of the Bastia team attacked police officers outside the stadium, pelting them with stones and explosive devices
Interior Minister Bernard Cazeneuve called for "an end to this violent behaviour, especially towards security forces who must be respected."
In April the Upper Corsican municipality banned supporters of neighbouring club AC Ajaccio from going to Furiani stadium for a match against Bastia.
In other games over the weekend, newly-promoted Caen made a bright start to their return to the top flight with a 3-0 win at Evian.
The game was all but over by half-time after Ngolo Kante opened the scoring and Mathieu Duhamel added a brace.
Nice won a thrilling battle with Toulouse 3-2 at the Allianz Riviera.
Argentine forward Dario Cvitanich opened the scoring on 23 minutes but two goals in as many minutes just before the break from Denmark international Martin Braithwaite and Ben Yedder turned the game on its head.
Cvitanich equalised just after the hour mark, though, and then Alexy Bosetti's pinpoint header on 68 minutes gave the Cote d'Azur side three points.
A Mevlut Erding brace, the first from the spot and the second deep into stoppage time, gave Saint-Etienne a 2-0 victory at Cup winners Guingamp.
In other games there were 1-0 wins for Bordeaux at 2012 champions Montpellier and Nantes at home to newly-promoted Lens.
Lille and Metz, the Ligue 2 champions, played out a 0-0 draw.
Champions Paris Saint-Germain kicked things off with a 2-2 draw at Reims on Friday night.
Sweden star Zlatan Ibrahimovic scored a brace either side of Reims goals by Prince Oniangue and Antonie Devaux.
Lyon host Rennes and Lorient travel to Monaco.