Administrators are wiki users who have "sysop rights". The wiki software has few features which use is restricted, but they are quite important.
What sysops can do
Sysops can edit protected pages and protect and unprotect pages. Protection of a page means that a non-sysop cannot modify it.
As a result, sysops can edit the protected pages in the MediaWiki namespace, which contain the messages that appear at the top of pages such as the Special:Whatlinkshere and the page that a blocked user will see when they try to edit a page.
Sysops can delete pages and their history. They can also view and restore deleted pages and their history.
Sysops can also permanently delete images. This is a non-reversible change: once deleted, always deleted. Note that there is no particular reason that image deletion should not be reversible; this is simply the way the software works at present.
Sysops can revert pages quickly. When looking at a user's contributions, a link that looks like this: [rollback] appears next to edits that are at the top of the edit history. Clicking on the link reverts to the last edit not authored by that user, with the edit summary (Reverted edits by X to last version by Y). This expedites the reversion of edits by anonymous vandals. Note, however, that all users, including those who are not logged in, can revert pages, only less conveniently.
Sysops can hide vandalism from the Recent Changes page. To do this, add &bot=1 to the end of the url used to access a user's contributions. For example:
When the rollback links on the contributions list are clicked, both the revert and the original edit that you are reverting will be hidden from the default Recentchanges display. This mechanism uses the marker originally added to keep massive bot edits from flooding recentchanges, hence the "bot". These changes will be hidden from recent changes unless you click the "bots" link to set hidebots=0. The edits are not hidden from contribs, history, watchlist, etc. The edits remain in the database and are not removed, but they no longer flood Recentchanges. The aim of this feature is to reduce the annoyance factor of a flood vandal with relatively little effort.
Block and unblock
Sysops can block and unblock IP addresses. IP blocks expire after 24 hours. Sysops can also block usernames. Special:Ipblocklist lists currently blocked addresses and usernames. Username blocking is not enabled by default. It can be switched on by setting $wgSysopUserBans=true; in LocalSettings.php. When it is not switched on, you will get an error message telling you the user does not exist.
Sysops can run read-only queries on the database.
Users with "bureaucrat" status can turn other users into sysops (but not remove sysop status). Bureaucrats are created by developers.
Users with ordinary access, including visitors who haven't "signed in", can still do many things, including the most important: editing pages and helping with maintenance tasks. But only signed-up users can upload files or rename pages.