The Publish All feature allows you to republish everything in your site or only items that are currently published. You can also republish the entire site to the preview environment. These features typically come in handy during development when you've been working only in the preview environment and you're ready to take a site live.
At the top of the Publish All dialog you'll see your site's statistics, that is, how many pages are unpublished (or not yet published), published, and dirty (items with changes that haven't been puslished).
When you republish all, only those items whose status is currently "Published" or "Published then modified" (a.k.a. "dirty") will be republished.
Publish all to preview will republish all content in your site, excluding trashed content, to the Preview server.
Publish all will send all content, excluding trashed content, to the live server. Be very careful with this method because it will also publish any items that are currently marked as unpublished. Trashed content will not be published.