The biggest shake up of Trump's Cabinet since he took office in January 2017 was announced by the president on Twitter as his administration works toward a potential meeting with North Korean leader Kim Jong Un after months of harsh rhetoric and rising tensions on Pyongyang's nuclear program.

The rare firing of the United States' top diplomat capped months of friction between the Republican president and the 65-year-old former Exxon Mobil Corp chief executive.

The tensions peaked last autumn amid reports Tillerson had called Trump a "moron" and considered resigning. Tillerson never denied using the word.

Critics expressed dismay at the decision to swap out top diplomats so soon before the unprecedented meeting and worried that Pompeo would encourage Trump to scrap the 2015 Iran nuclear deal and be hawkish on North Korea.

The move would sow more instability in the volatile Trump administration and marks the departure of another moderate who sought to emphasise the United States' strong ties to its allies amid Trump's criticism, critics said.

Trump announced the changes in a morning Twitter post and later told reporters more about why he removed Tillerson.

"We got along actually quite well but we disagreed on things," he said.

"When you look at the Iran deal: I think it's terrible, I guess he thinks it was OK. I wanted to break it or do something, and he felt a little bit differently."

At the State Department, a visibly emotional Tillerson said Trump called him around noon from Air Force One, hours after he was summarily dismissed via Twitter. Tillerson also spoke with White House Chief of Staff John Kelly.

He said his tenure ends on March 31 but he would delegate his responsibilities to John Sullivan, deputy secretary of state, at the end of Tuesday.

He pointedly declined to thank Trump personally or praise him, as he has done on previous occasions, but emphasised his strong relationship with Secretary of Defence Jim Mattis. Together, the two were seen as a moderating influence on some of Trump's policies.

Pompeo, a former Army officer who represented a Kansas district in the House of Representatives before taking the Central Intelligence Agency job, is seen as a Trump loyalist who has enjoyed a less hostile relationship with career spies than Tillerson had with career diplomats.

Trump chose the CIA's deputy director, Gina Haspel, to replace Pompeo there. A veteran CIA clandestine officer, Haspel is backed by many in the US intelligence community but is regarded warily by some in Congress for her involvement in the agency's "black site" detention facilities.

Senior White House officials said Trump wanted his new team in place before any summit with Kim, who invited the US president to meet by May.

State Department officials said Tillerson did not know why he was being pushed out and had intended to stay. Tillerson joins a long list of senior officials who have either resigned or been fired since Trump took office.

He had emerged as a vocal critic on Russia - for its role in the annexation of Crimea, support for Syrian President Bashar al-Assad and alleged meddling in the U.S. election.

Australian Associated Press