Decaflon

I can't mention the parties involved here but my employer is having a hell of a time with their hosting company and since I'm the lone developer, I have to figure out the right solution.

We were moved to a new server last week because the old server we were on was occasionally crashing, more than once each week. The host said it had hard drive issues so they moved the sites. This server has 3 sites that all receive very high traffic.

Since we moved we started seeing issues with the sites running slow. There were lots of things that happened during this time so it's hard to tell exactly where the problem was. I'll try and lay out the different issues:

- one of our sites doubled in traffic. it is a PHP/MySQL site and has zero caching. It was developed externally and I told the developers to cache the pages. It should be done by next week.

- we moved to a new advertising service and to make the management of adds easy, I wrote some scripts with PHP to use a MySQL database and serve up the ads. At first this slowed the server down a lot so I cached all the scripts so that they wouldn't have to do any queries. These files currently look like this:

<?php
if(file_exists...)
include('xxx');
exit;
?>

2 of our sites use these scripts, they make up about 80% of the traffic.

- I added a feature to one of the sites that meant an extra database request on some of the highest-traffic pages for that site. We then had some issues with performance so as an extra measure I implemented Perl's MemoryCache module (this is a Perl/MySQL site). So I was caching some of the results of various functions in memory and these caches were set to last for an hour. I was also auto-purging every hour.

Last Monday our server had 1 gig of RAM. the host recommended we increase that... now we have 3 gigs. We were at 90% memory consumption when we had 1 gig. Today, with 3 gigs, we suddenly hit 99% consumption. The sites have been slow all day.

In the morning the host was saying that it was MySQL eating up all the memory. In the afternoon, however, they said we had too many PHP and CGI processes running. They recommended upgrading to a server with a more powerful CPU.

I can't do much on my end to verify any of this because we have a "managed" dedicated server, which means I can't do much on the command-line. It's very frustrating, and the host's support is not very good.

So, um, what do I do? Is this thing about the PHP/CGI processes really true? I know the server doesn't have a very good setup (no FastCGI). Would changing the ad system to use static HTML files improve things (since most pages have on average, 3 iframes, each running a separate PHP file)? Should I just assume that our host is problematic and look for hosting elsewhere (I'm leaning toward this right now)? At least if we were on a better server with root access, I could setup PHP/CGI better and get more performance.

I'm just looking for any advice here... this whole mess is keeping me from doing *real* work.

p.s. this is an Apache Linux server.