The immediate trigger was the emergence of two new cases of alleged American spying. They inflamed a furor that erupted last year when it was learned that the U.S. was intercepting Internet traffic in Germany and eavesdropping on Chancellor Angela Merkel's cellphone calls.
More broadly, the move to expel the CIA station chief appears to reflect a Germany out of patience with what it sees as a pattern of American disrespect and interference.
"The representative of the U.S. intelligence services at the United States Embassy has been asked to leave Germany," German government spokesman Steffen Seibert said in a statement.
"The request occurred against the backdrop of the ongoing investigation by federal prosecutors as well as the questions that were posed months ago about the activities of U.S. intelligence agencies in Germany," he added. "The government takes the matter very seriously."
U.S. officials described Germany's action as extraordinary.
While agents have been expelled from time to time, usually by unfriendly powers, a former U.S. official said he couldn't remember an instance since the end of the Cold War in which the ranking intelligence official was asked to leave a country.
The official spoke on condition of anonymity because he wasn't authorized to discuss intelligence issues publicly.
Germany refused to identify the CIA station chief by name. In the United States, it is illegal to disclose the name of an undercover operative.
Shortly before Thursday's announcement, Merkel told reporters that Germany and the United States had "very different approaches" to the role of intelligence agencies, and she insisted that any spying on allies is "a waste of energy."
White House press secretary Josh Earnest wouldn't comment on Germany's decision but said the U.S. takes intelligence matters "very seriously."
"I don't want you to come away from this exchange thinking we take this matter lightly," he said, adding that the U.S. and Germany continue to have a strong partnership.
The reports last year that the National Security Agency had targeted Merkel and Internet traffic have triggered a German criminal investigation and a parliamentary probe.
On Wednesday, German police raided properties in the Berlin area in what Seibert said was a case involving "a very serious suspicion" of espionage.
German media reported that the man being investigated worked at Germany's Defense Ministry in a department dealing with international security policy, and he had aroused suspicion because of his close contacts to alleged U.S. spies.
Last week, a 31-year-old German intelligence employee was arrested on suspicion of spying for foreign powers since 2012. German media have reported that he spied for the CIA.
German Foreign Minister Frank-Walter Steinmeier said he could not comprehend why the U.S. would spy on his country.
"We speak to each other all the time, and nobody keeps their views secret," he said in an interview published Wednesday by the Saarbruecker Zeitung.
Interior Minister Thomas de Maiziere said the political fallout appeared to outweigh any harm done to Germany by the alleged spying.
"If the situation remains what we know now, the information reaped by this suspected espionage is laughable," de Maiziere said in a statement. "However, the political damage is already disproportionate and serious."
Under the elaborate rules of international diplomacy, Germany's move to kick out the spy chief was a request two steps short of a formal expulsion.
In the past two years, Germany has also asked diplomats from Syria and Pakistan to leave the country.
Geir Moulson in Berlin, and Nedra Pickler and Ken Dilanian in Washington contributed to this report.
Follow Frank Jordans on Twitter at http://www.twitter.com/wirereporter
Copyright 2014 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.