The GUI needs GTK 1.2.x or GTK 2.0 (it isn't fully GTK, but the panels are), so GTK (and the devel stuff, usually called gtk-dev) has to be installed. You can build it by specifying --enable-gui during ./configure. Then, to turn on GUI mode, you have to execute the gmplayer binary.

In order to use the GUI in a comfortable way and as intended, you are strongly advised to install a skin. See Skins at the download page. A skin should be extracted to the usual system-wide directory $PREFIX/share/mplayer/skins or to the user specific directory $HOME/.mplayer/skins and resides as a subdirectory there. MPlayer by default first looks in the user specific directory, then the system-wide directory for a subdirectory named default (which simply can be a link to your favourite skin) to load the skin, but you can use the -skin myskin option, or the skin=myskin or gui_skin=myskin config file directives, or the skin browser to use a different skin from the skins directories.