Problem with virtualmin apache

Hello,
I have problem with webserver Apache… I am owner of highly visited website (~ 100 000 UIP/day), but apache does not want to accept connections, so 95% of users gets error message Connection Timeout…

there are no apache errors in error log, neither there are no kernel errors.

There is my apache configuration file:
http://pastebin.com/m58178501

Anybody knows what to do, to solve this?

Are you sure that you aren’t running into DNS issues?

Do you see any errors if you go to intodns.com, and enter one of the domain names you’re having trouble with there?

-Eric

No, it is not caused by DNS, because when I try to access direct server IP, it does not work too…

Also when I disable Apache server load is about 0.9, but when Apache is enabled - it creates 500 processes, and server load raises up to 416.33

Aha, ok.

It sounds like you may have a scaling problem then :slight_smile:

Apache – or one of the dependencies, such as PHP or MySQL, may be having trouble keeping up with the load.

Have things worked for you in the past with this server? If so, has anything changed recently?

While any number of things could be at fault, a common reason for this problem is a slow MySQL query.

You might consider enabling the “log_slow_queries” in the my.cnf file.

Also, the “mytop” program is something you could install and run to get an idea of what processes are running within MySQL – you could use that to track down what’s taking up so much time.

-Eric

So, we have no data to go on here, so anything we say about how to scale your application would be wild-ass guessing. And guessing just wastes everybody’s time, so let’s figure out what is causing slowness before trying to fix it.

Does top give you any clues about what process is most demanding?

No clues in the error_log? Timeouts or anything like that?

Eric’s suggestions about MySQL are also worth doing.

Nothing - no log error messages.
Server worked well within last week. There were done no changes in HW or configuration.

But there is too much connections in state SYN_RECV - about 2000. When I turn on firewall, this number goes down to 300, but no change in speed of connection.

There is Apache status: http://www.world-banking.info/server-status

This is really wierd problem, I have already read and tried all tutorials & help suggestions, that I have found at google, but nothing helped…

You’re still not showing us the information we asked for. Is there anything in top eating CPU? Do the MySQL tools Eric mentioned give you any information about what queries are slow and where the work is going?

I seem to recall that you had some performance problems a couple of weeks ago, and they all boiled down to problems in the way the application was using MySQL and the lack of indexes on very heavily used large tables. I’m guessing this is more of the same.

Oh, and I’ll mention that your server-status shows that Apache is hitting its configured server limit…and I’m guessing this is happening because the application is slow. Apache is doing exactly what it’s supposed to do, but something is making requests take longer to complete than they should…probably database issues.

Apache takes 10%; MySQL takes about 8% of CPU; slow query log is turned on and slow query time is set to 1 sec., the slow query log is clear - no slow queries recorded…

I remember those 2 sites on that server and after troubleshooting a lot of stuff, those sites were screaming fast.

I would now have a tendency to look into the network and per haps even the network of your datacenter. Per haps a bad switch / router, per haps iface or you are on 100mb line and need a 1gb, you know stuff like that.

a traceroute:

  1.  1.69ms  	1.53ms  	1.54ms  	nameintelligence.com  	 Reverse IP | Ping | DNS Lookup |
    
  2. 1.68ms 1.36ms 1.66ms ip-64-246-162-161.ipd.ccom.net Reverse IP | Ping | DNS Lookup |
  3. 1.65ms 1.43ms 1.68ms 216.145.30.190 Reverse IP | Ping | DNS Lookup |
  4. 1.67ms 1.27ms 1.61ms 19b1-19pe1-vlan259.sea.fibercloud.net Reverse IP | Ping | DNS Lookup |
  5. 1.67ms 1.37ms 1.68ms ge-6-17.car4.Seattle1.Level3.net Reverse IP | Ping | DNS Lookup |
  6. 14.65ms 2.42ms 7.68ms ae-32-52.ebr2.Seattle1.Level3.net Reverse IP | Ping | DNS Lookup |
  7. 12.67ms 2.34ms 1.62ms ae-1-100.ebr1.Seattle1.Level3.net Reverse IP | Ping | DNS Lookup |
  8. 18.70ms 18.33ms 27.68ms ae-7.ebr2.SanJose1.Level3.net Reverse IP | Ping | DNS Lookup |
  9. 18.66ms 19.36ms 20.66ms ae-62-62.csw1.SanJose1.Level3.net Reverse IP | Ping | DNS Lookup |
  10. 19.59ms 	48.55ms 	19.68ms 	ae-4-99.edge2.SanJose3.Level3.net 	Reverse IP | Ping | DNS Lookup |
    
  11. 20.62ms 	20.25ms 	21.66ms 	xe-4-2-0.sjc10.ip.tiscali.net 	Reverse IP | Ping | DNS Lookup |
    
  12. 186.60ms 	187.26ms 	187.64ms 	so-0-0-0.prg11.ip.tiscali.net 	Reverse IP | Ping | DNS Lookup |
    
  13. 188.63ms 	189.03ms 	189.57ms 	master-internet-gw.ip.tiscali.net 	Reverse IP | Ping | DNS Lookup |
    
  14. 189.63ms 	190.20ms 	190.61ms 	praha-4d-c1-vl272.masterinter.net 	Reverse IP | Ping | DNS Lookup |
    
  15. * 	* 	* 	Request Timed Out 	
    
  16. * 	* 	* 	Request Timed Out
    

Do you have your ISP IP’s in your network settings? Cause the trace can reach your ISP but not your box.