WordPress is a miracle itself, with the ease of the installation process and the user-friendly interface and everything, but it comes quite bare.
If you want impressive performance, easy user interaction, good search engine optimization you do need some plugins.

Akismet
Akismet is a plugin to protect you from spam comments. It comes with the default installation, but you have to register and API key. You can choose the amount of money you want to donate; please be generous, they’ll really catch unwanted comments.
Plugin site: http://wordpress.org/extend/plugins/akismet/
WordPress SEO
This is far the most complete and easy to use SEO plugin provided by Joost de Valk. It help removing unnecessary meta tags, taxonomy bases from ULRs, adds support for SEO titles, and so on. Plugin site: http://wordpress.org/extend/plugins/wordpress-seo/
My current configuration for import as a ZIP file.
Google Analytics for WordPress
This plugins was developed by the same Joost de Valk as the first-rate SEO plugin. With this, you can easily add Analytics to your site. Far more easy to manage and install than the others.
Plugin site: http://wordpress.org/extend/plugins/google-analytics-for-wordpress/
Hyper Cache
There are really good all-in-one cache plugins for WordPress, Hyper Cache only adds page caching support, but it is probably the fastest and the most lightweight on this topic.
Plugin site: http://wordpress.org/extend/plugins/hyper-cache/
WP-Paginate
The default pagination in WordPress is not good, it just simply lack any good features, like pagination itself. Prev/next is not pagination, at least no anything like this plugin adds.
Plugin site: http://wordpress.org/extend/plugins/wp-paginate/
TinyMCE Advanced
Adds a configuration possibility of the Visual editor’s capabilities, for example tables, advanced images, etc.
Plugin site: http://wordpress.org/extend/plugins/tinymce-advanced/
WP to Twitter
Really easy to install and easy but widely configurable plugin to automatically post content to twitter as well. You can even specify the text that should be tweeted.
Plugin site: http://wordpress.org/extend/plugins/wp-to-twitter/
Ultimate Post Type Manager
As of WordPress 3.0 you are able to define custom post types beside the original Page and Post, for example Products or Poems or anything. This plugins add GUI for easy extending.
Plugin site: http://wordpress.org/extend/plugins/ultimate-post-type-manager/
Ultimate Taxonomy Manager
WordPress 2.3 gained the ability to add custom relation types (taxonomy) into your side next to Category and Tag. This plugins add GUI for easy configuration.
Plugin site: http://wordpress.org/extend/plugins/ultimate-taxonomy-manager/
WordPress MU Domain Mapping
This plugin differs from the others. First of all, this is only for WordPress Networks. Second: after the installation and activation for the whole network, you need to copy a file, to modify some and to configure the mapping for the domains. A very good tutorial was made by Otto on using Domain Mapping plugin.
Plugin site: http://wordpress.org/extend/plugins/wordpress-mu-domain-mapping/

Of course, my list is not complete. There are other very good plugins as well a lot I’ve never tried yet. I’m intending to update this list whenever I find something vital again.