How does Java's PriorityQueue differ from a min-heap?

Why did they name PriorityQueue if you can't insertWithPriority? It seems very similar to a heap. Are there any differences? If no difference, then why was it named PriorityQueue and not Heap?