Murray cut a frustrated figure as he was swept aside by Borna Coric for the second time in the pair's four meetings.
Rafael Nadal extended his flawless record on clay this season to 12-0 with a 6-3, 6-1 destruction of Nick Kyrgios.
And defending champion Novak Djokovic also eased into the quarter-finals with a 6-4, 7-5 win over Feliciano Lopez.
Coric was only handed a late reprieve to enter the draw in Madrid after losing in qualifying to Mikhail Kukushkin.
"After one match of the quallies where I lost and I was booking my flight back to home," said Coric.
"I couldn't imagine this when I woke up this morning, I always believed but I wasn't sure how I was going to play.
"In the end I played as I had dreamed."
However, Murray cut a frustrated figure as he was swept aside by the 20-year-old Croatian for the second time in the pair's four meetings.
"I didn't help myself find a way into the match to start playing better," said Murray.
"I definitely think I need to be concerned about today.
"It's not always the worst thing losing a match, but it's sometimes the manner of how you lose the match is what can be concerning or disappointing."
Coric broke the fragile Murray serve three times in the first set under the roof at the Manolo Santana centre court due to rain in the Spanish capital.
And one poor service game from the Wimbledon champion cost him dear in the second as Coric served out for the match to set up a quarter-final meeting with Dominic Thiem.
Nadal had no such troubles as he dominated Kyrgios to beat the Australian in just over an hour.
"I played really well tonight. Right from the start I made very few errors," said the Spaniard.
"To win 6-3, 6-1 against an opponent as good as Nick, isn't normal."
The 14-time Grand Slam champion had a slice of fortune as a let cord and a Kyrgios double fault handed him the two breaks he needed to take the first set.
However, Kyrgios showed little resistence after being broken once more in the first game of the second set as he was whistled from at the end by Spanish fans unhappy at only seeing Nadal on court for 73 minutes.
And he admitted to having a defeatist attitude from the start.
"I wasn't expecting to beat him at all tonight, to be honest," said Kyrgios.
Nadal faces David Goffin in Friday's quarter-finals after the Belgian beat world number six Milos Raonic 6-4, 6-2.
After suffering his own struggles of late, Djokovic was encouraged by his display in seeing off Lopez and a partisan crowd to reach the last eight.
"If you're playing against a Spanish player in Madrid, it's obviously a different game, it's a different feeling," said Djokovic.
"They have a lot of support. You need to kind of hold your nerve, hold your concentration, and use all the necessary energy at the right moments."
Djokovic faces a much tougher test on Friday against world number eight Kei Nishikori, who earlier swept aside David Ferrer 6-4, 6-3.
However, Nishikori has only won two of their previous 13 meetings.
"Novak is someone that I don't have good record (against)," said the Japanese.
"I got to stay really positive and I got to play great tennis to beat him. I try to enjoy my big challenge tomorrow."
Thiem saved five match points to edge out Grigor Dimitrov a thrilling third set tie-break 4-6, 6-4, 7-6 (11/9).
Twenty-year-old German Alexander Zverev continued his fine form with a 6-4, 6-4 win over 11th seed Tomas Berdych to set up a quarter-final against Pablo Cuevas, who beat Benoit Paire 7-5, 0-6, 6-1.
In the women's Madrid Open, third seed Simona Halep remains on course to defend her title as she thrashed American Coco Vandeweghe 6-1, 6-1.
Halep faces Latvian Anastasija Sevastova in Friday's semi-finals after the world number 22 saw off Kiki Bertens 6-3, 6-3.
Svetlana Kuznetsova ended Eugenie Bouchard's impressive run to the quarters in convincing fashion 6-4, 6-0.
The Russian will meet Kristina Mladenovic in the last four as she beat Sorana Cristea 6-4, 6-4