Eight men were initially detained, including three Austrians "with a migration background"
The raids however were not related to the arrest in Vienna last week of a 17-year-old suspected Islamist extremist, and investigators did not believe a terror attack was imminent, reports said.
"As part of an ongoing investigation into suspected membership of a terrorist organisation ('IS') a coordinated operation planned for some time took place involving 800 officers," Graz prosecutors said.
Eight men were initially detained, including three Austrians "with a migration background", two Bosnians, a Syrian, a Bulgarian and a Macedonian, all aged between 21 and 49, a spokesman said.
Four of the arrests were in Vienna and four in Graz.
A further three suspects from the Balkans were later also arrested in Graz.
Media reports said that as well as apartments, police also raided unofficial mosques, mostly in Graz. They also said the operation targeted individuals from the former Yugoslavia suspected of establishing a jihadist network in Austria.
The Kronen Zeitung tabloid reported that the raids focused on the network of a Muslim preacher from Bosnia jailed for 20 years last July in Graz for recruiting young IS fighters.
The accused, known as Ebu Tejma, "brainwashed" dozens of people aged between 14 and 30 and enlisted a number of them to fight for IS in Syria, his trial heard.
Tejma fled from Bosnia to Vienna following the break-up of Yugoslavia and preached in various Austrian and southern German cities, becoming a "key figure" in pushing IS propaganda, according to the prosecution.
He was arrested during a far-reaching crackdown on Austrian jihadist networks in 2014 and denied the charges against him.
Austria has so far been spared getting hit by the string of attacks by Islamist extremists in other European countries in the past few years.
But around 300 people have either left or were intercepted trying to leave Austria to fight in Syria, according to the interior ministry. Around 40 have died there while some 90 have come back.
The 17-year-old of Albanian origin arrested in Vienna on Friday was thought to have been planning a bomb attack. The following day a suspected accomplice was arrested in Neuss, Germany.
Media reports said that they had experimented with making explosives in the 21-year-old's Neuss apartment. Authorities said the Austrian had told investigators he made an experimental bomb.