"We are used to making history, we have big ambitions and the next big ambition is to beat Sweden", said boss Vladimir Petkovic who will now look to end 64 years of hurt by matching Switzerland's record showing by progressing through to the quarter-finals.
Swedish coach Janne Andersson declined to answer questions whether his team has been underestimated at the World Cup after a scrappy win over Switzerland secured them a spot in the quarter-finals. He was able to get a shot off on the turn, which sprung the Swiss keeper into life as he got down well to make a good save.
The breakthrough came when Toivanen picked out Forsberg on the edge of the area.
While that might offend the purists, Sweden midfielder Albin Ekdal has already responded to criticism that his side are not the most easy on the eye.
Sweden have been playing as organized and team-oriented soccer as anyone at the World Cup this year, and they will need to be just as sound in all facets again on Tuesday.
Sweden had one attempt fewer on target, with the only difference being that one effort took a heavy deflection and wrong-footed Swiss goalkeeper Yann Sommer to send the Swedes into the next round.
Overall, though, Switzerland are still a team very much on the up.
This was another opportunity spurned by the Swiss, who have reached the last 16 in four of the last five World Cups only to be eliminated without scoring a goal.
Southgate recognises golden World Cup opportunity for England
Yet Eriksson does not consider that a major issue given he thinks the team's strength to be how it functions as a unit. But the Selecao were stunned by the first European side to qualify for this world cup - the Red Devils of Belgium.
Trump Administration Rescinds Obama-Era Guidance Encouraging Affirmative Action
It was one of the largest joint actions ever taken by Asian-American organisations in pursuit of equal education rights. The new affirmative action guidance could add to an already contentious fight over the next justice.
Kieran Trippier revels in England’s penalty shootout victory over Colombia
However, if Les Bleus can continue the forward momentum they showed against Argentina, they will take some stopping. When you have two world-class players connecting things all over the park, you have a chance to go far.
Moments later the Swede's nearly had a second. He was probably their best player in Russian Federation, alongside Xherdan Shaqiri, while Granit Xhaka helped them dictate the tempo of at least part of every game.
Their 10 corner nearly ended in a goal but Andreas Granqvist - the Rock of Sweden and best defender of the tournament - cleared the danger.
Steven Zuber got Switzerland's goal against Brazil and SportPesa offer 12/1 (13.0) that the Hoffenheim man notches the opening goal here, while the same firm go 5/1 (6.0) that Zuber scores at any time.
The Swiss have twice conceded the opening goal in Russian Federation - first in their 1-1 draw with Brazil and again against Serbia - when Switzerland became the first team at this World Cup to win after going a goal down.
Sweden soon, made series of substitutions to minimise the pressure and the move proved to be successful as they were close to earn a penalty in the dying minutes of the game.
Referee Damir Skomina awarded Sweden a penalty in stoppage time for a foul by Michael Lang on Martin Olsson but overturned his own decision after a review showed Lang's push was just outside the area. Sommer parried away Toivonen's curler and the final whistle seconded seconds later to the joy of Sweden.
Switzerland pushed for an equalizer but Sweden closed ranks at the back, keeping the sixth-ranked side in the world at bay to record their third clean sheet of the tournament.