Perez, whose opener at a wet Camping World Stadium in Orlando, Florida, had been cancelled out by Juan Carlos Arce, scored the winner in the 87th minute for the Copa debutants.
The veteran Vancouver Whitecaps striker capitalised on good work from Abdiel Arroyo to secure Panama's win in the Group D clash.
Perez had put Panama ahead with a well-deserved opener in the 11th minute as they made a bright start.
However, Bolivia worked their way back into the encounter and levelled through Arce, but their losing run was still stretched to five games.
Both teams made numerous changes to the teams fielded in their final friendlies prior to the Copa.
For Panama, it included starts for Perez and Anibal Godoy, while Bolivia opted for Carlos Lampe in goals ahead of Guillermo Vizcarra.
It was Panama who started the better with Gabriel Torres particularly dangerous, firing a third-minute effort wide after a corner fell to him outside the area.
The Zamora attacker then forced Lampe into a save as the out-of-position goalkeeper hurried back to tip the long-range free-kick over.
The early pressure would eventually lead to the opener in the 11th minute through Perez, who struck his 40th international goal.
Alberto Quintero was played through down the left and crossed into the area, where a desperate Perez put his finish into the bottom corner.
That brought a response from Bolivia as Arce headed a cross on target before Jaime Penedo was called into action again soon after, pushing away a Yasmani Duk effort.
Perez threatened again approaching the half-hour mark, but his flick at the near post was saved down low by Lampe.
Driving rain greeted players at the start of the second half and Bolivia found their equaliser in the 54th minute.
A set-piece was poorly cleared by Harold Cummings and the ball fell to Arce to smash into the roof of the net with his left.
Cummings would somewhat make up for that error with a last-ditch tackle on Duk before a fine team move almost led to Panama regaining the lead.
Armando Cooper picked out Quintero and his strike on the turn was well saved by Lampe down low to his right.
Going in search of a winner, Luis Tejada somehow put a glorious opportunity over the bar from inside the area in the 85th minute.
But a goal would follow just minutes later as substitute Arroyo made space down the right before squaring for Perez.
There was still further late drama as a penalty shout was turned down for Bolivia as Panama held on.