A 27-year occupation followed before Okinawa was given back to Japan, but US bases remained.
But the move is unlikely to reduce frustration on the strategic island that remains crowded with American bases -- a legacy of World War II.
The agreement covers about 4,000 hectares (9,900 acres) of the 7,500-hectare Camp Gonsalves jungle warfare training centre, also known as the Northern Training Area.
It marks "the largest land return to the Japanese government" since Okinawa reverted to Japanese control in 1972, Lieutenant General Jerry P. Martinez, US Forces Japan Commander, said in a statement.
"It will reduce the amount of US-administered land on Okinawa by 17 percent," he added.
But the area being handed back is in a sparsely populated region, while other facilities, including air bases, are located farther south in or near crowded residential areas of the small island with a population exceeding one million people.
The US wrested Okinawa from Japan in the closing months of the war. A 27-year occupation followed before Okinawa was given back to Japan, but US bases remained.
The island makes up less than one percent of Japan's total area but accounts for about 71 percent of land allotted for US bases in the country after the handover. It accounted for about 75 percent before.
Today Okinawa is considered a strategic linchpin for security allies Japan and the US as they face China's increasing military might and unpredictable North Korea's development of nuclear weapons and long-range missiles.
It hosts more than half of the approximately 47,000 American military personnel stationed in Japan.
Okinawans have complained for decades that the rest of Japan ignores their burden in supporting the alliance with the US.
Japanese Prime Minister Shinzo Abe, who has fought with Okinawan Governor Takeshi Onaga over base issues, welcomed the agreement, the details of which have been known for months.
"I express gratitude towards all relevant people in Japan and the US who tackled this difficult task," he said, adding the returned land will contribute to Okinawa's development.
Caroline Kennedy, the US ambassador, called it a "milestone".
Onaga, a fierce critic of the US military, planned to boycott a ceremony in Okinawa on Thursday marking the handover, reportedly attending a local protest rally instead.
Crimes and accidents by military personnel, civilian employees and dependents on Okinawa, combined with noise and inconvenience from military flights and training, have frustrated residents.