Federal law says the government has to deliver the mail. It would take an act of Congress to change that.
Because federal law establishes the postal service as a core function of the government, privatizing USPS would require Congress to pass a new law.
The U.S. Postal Service in its current form was established by the Postal Reorganization Act of 1970.
That law clearly states that the mail has to be delivered by the federal government, not private companies: “The United States Postal Service shall be operated as a basic and fundamental service provided to the people by the Government of the United States, authorized by the Constitution, created by Act of Congress, and supported by the people.”
The law also lays out the fundamental requirements of that service, such as delivering six days a week and delivering to every address in the country, no matter how remote and expensive it is to reach.
Because these functions are enshrined in federal law, the president cannot change them with executive orders.