IP promises almost nothing
IP will try to deliver your packet. It may drop it, deliver it twice, deliver it out of order, or corrupt it. There is no acknowledgement and no retry. That is called best-effort delivery, and it is deliberate — keeping the network simple is what let it scale to the whole planet.
Everything anyone wants from a network beyond "try to deliver this" is built on top, at the transport layer, on the two end machines. That is where TCP and UDP live.
What a port is
Both protocols add port numbers. IP gets a packet to the right machine. The port gets it to the right program on that machine.