That could be several reasons:
1.- Lots of ads (About or more than 100.000 ads on the site, which doesn't seems to be the case)
2.- Some plugin is eating the memory or doing a slow sql query
/**
* Copy this code to config.php file
*
* This will show SQL queries at the bottom of the site
*/
define('OSC_DEBUG_DB', true) ;
to know if some SQL queries are slower from what they're supposed to be
3.-Is the cron getting executed correctly? Go to admin > Settings > Cron System and disable it (temporally, to check load speed)
4.- Use Firefox + firebug or chrom + developer console to know is external resources are slow
Maybe some external script/file is making your site slow.
Thanks