Third-ranked France had been bidding to seal their last 16 berth along with champions Japan after a 1-0 win over England in their opener, but must now wait until their final game in a tight Group F.
Colombia, ranked 28, moved top of Group F with four points from two games, with France on three, as England and Mexico play later Saturday.
“This is an ambitious team, eager and willing to go after victory,” said Colombia coach Fabian Taborda.
“We're not here just to make up numbers, we're here to improve and win. These ladies are fighters and warriors.”
The French fell behind early when Andrade, 23, picked up a pass from Yoreli Rincon after 19 minutes to beat Sarah Bouhaddi in the France goal.
That strike visibly boosted Las Cafeteras' confidence, as French morale took a bashing.
And late in the game, with all the French team camped in the Colombian half pushing for an equaliser, second-half substitute Usme broke through to close out the victory three minutes into injury time.
“Disappointment is the only word for it,” said France coach Philippe Bergeroo.
“Today Colombia really deserved their victory. We knew they were very good, but when you come up against an enthusiastic team like that it's difficult.
“And then the morale took a knock when we conceded the first goal.”
It was a first World Cup win for the Colombians who drew 1-1 with Mexico in their opener.
“We came up against a super goalkeeper and lacked finesse in our technique,” said Bergeroo.
“At the end of the match we tried to go all out and we conceded the second goal.
“For me it's the team who lost the match today not any individual player. We missed enormous chances. We can only blame ourselves.”
Colombian goalkeeper Sandra Sepulveda, who missed their opening game because of injury, pulled off a string of fine saves.
France striker Eugenie Le Sommer had Les Bleues' best chance minutes before the break, but Sepulveda held firm, as she also pulled off magnificent saves from Elise Bussaglia (46) and Claire Lavogez (90+1).
“It's a tremendous joy and gratifying to be part of this great moment,” said Sepulveda.
“We know how much we've trained to win this match. Our work has borne fruit.”
France almost got the equaliser with 20 minutes to go when Sepulveda handled the ball just outside of her area, but Camille Abily's tame effort went straight into the Colombian goalie's hands.
In desperation Bergeroo sent in his three substitutes in a bid to boost the front line.
Abily, Le Sommer, Lavogez, Jessica Houara and Amandine Henry all came close to earning a point, but Colombia's backline held firm.
Las Cafeteras kept the French at bay and their composure paid off when Usme's quick thinking paid off, with wild celebrations moments later at the final whistle.
“I think people think that Colombia come here just to compete,” said Andrade of her side who are competing in their second World Cup.
“We're very united as a group, it's very difficult for us to be defeated,” she warned.
“We'll continue surprising I'm sure. We've evolved in our mindset and are now thinking big.”
The result leaves it all to play for in Wednesday's final group games with France, fourth at the 2011 World Cup and 2012 Olympics, taking on Mexico and Colombia playing England.
The top two teams in each group advance to the knock-out round along with the four best third-place finishers.