apache issue??

When I log in to my server as root on port 10000 the default system status shows a red X for apache web server. When I push the green arrow to its right I get:

Failed to start service :

Starting httpd: (98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
[FAILED]

Now in fact apache IS running and serving pages just fine. What can I do to get the status report back on track? VM Pro3.81

Dave

It’s possible the Apache pid files are out of sync with what Virtualmin is expecting. Using the command line, you may want to restart both Apache and Virtualmin.

If you aren’t sure of what commands to use, I can help, just let me know what distro you’re using.

-Eric

using centos 5.5

Okay, so you can try running these two commands as root on your server:

/etc/init.d/httpd restart /etc/init.d/webmin restart

After doing that, do the services show up correctly in the status area of Virtualmin?

-Eric

[root@cserver admin]# /etc/init.d/httpd restart
Stopping httpd: [FAILED]
Starting httpd: (98)Address already in use: make_sock: could not bind to address [::]:80
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs

That looks to me like it is trying to stop the Apache, but fails, and the subsequent start attempt fails because the port is already in use (Apache still running?)

Try ps aux | grep apache to see if there are any Apache processes currently running.

Also check the file /var/run/apache2.pid which should contain the process ID if any Apache is running currently.

[root@cserver admin]# cat /var/run/apache2.pid
cat: /var/run/apache2.pid: No such file or directory
[root@cserver admin]# cd /var/run
[root@cserver run]# ls
acpid.socket gdm.pid mysqld saslauthd
atd.pid gpm.pid named sdp
audispd_events haldaemon.pid named.pid setrans
auditd.pid hpiod.pid netreport setroubleshoot
autofs.fifo-misc hpiod.port NetworkManager snmpd
autofs.fifo-net hpssd.pid news spamassassin
avahi-daemon hpssd.port nscd sshd.pid
console klogd.pid ntpd.pid sudo
crond.pid lvm pcscd.comm syslogd.pid
cups mailman pcscd.pid utmp
cupsd.pid mdadm pcscd.pub winbindd
dbus mdmpd pm wpa_supplicant
ddclient messagebus.pid ppp xfs.pid
dhclient-eth0.pid mod_fcgid proftpd xinetd.pid
dovecot munin rpc.statd.pid

Okay, so there’s no Apache PID file. That would explain why stopping the service fails. What about the ps aux output I mentioned?

Next, you can check what is listening on port 80, if it’s not Apache, using this:

netstat -ptln | grep :80

And a hint: When posting output of commands, please embed the text in code-tags, like so: and put your text between those tags. That will prevent line-breaks from being ignored and will use a monospaced font, making text output MUCH easier to read.

Next hint: If netstat reveals that it is indeed Apache who is listening on :80, the PID file might have gotten lost. You can then kill all active Apache processes, and subsequently restart it, by doing this:

killall apache2
/etc/init.d/apache2 start
[root@cserver run]# ps -aux | grep apache
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
apache     586  0.0  1.0  45248 22512 ?        S    Oct17   0:00 /usr/sbin/httpd
apache     587  0.0  2.2  90368 45792 ?        S    Oct17   1:57 /usr/sbin/httpd
apache     588  0.0  2.1  90160 45636 ?        S    Oct17   1:51 /usr/sbin/httpd
apache     589  0.0  2.2  90436 45752 ?        S    Oct17   1:55 /usr/sbin/httpd
apache     590  0.0  2.2  90448 45772 ?        S    Oct17   2:00 /usr/sbin/httpd
apache     591  0.0  2.2  90348 46368 ?        S    Oct17   1:58 /usr/sbin/httpd
apache     592  0.0  2.2  90400 45824 ?        S    Oct17   1:47 /usr/sbin/httpd
apache     593  0.0  2.2  90332 46392 ?        S    Oct17   1:49 /usr/sbin/httpd
apache     594  0.0  2.2  90416 45848 ?        S    Oct17   1:57 /usr/sbin/httpd
root      8065  0.0  0.0   4876   692 pts/2    S+   16:27   0:00 grep apache

[root@cserver run]# netstat -ptln | grep :80
tcp        0      0 :::80                       :::*                        LISTEN      586/httpd           

and thanks for the [code] tip. I knew such a thing existed but didn’t know exactly what it was.

Question: what distro and what version of Apache are you running? In mine, the executable is named “apache2” and not “httpd”. /usr/sbin/httpd -v should reveal that info.

CentOS 5.5, apache 2.2.3. I think Debian and derivatives use apacheX as process name

killall httpd
/etc/init.d/httpd start

and the virtualmin system status page still shows apache as not running. Of course apache is serving up the page I’m looking at…

Dave

Maybe VMin uses an incorrect method to determine if Apache is running… Try “Refresh System Information” or System Settings / Re-Check Configuration. Might have to do with “httpd vs. apache2” naming, or with PID file.

Also check the items in the Apache Webmin module, Module Config / System configuration, like “Command to start/stop Apache” and the “Path to Apache PID file”.

the /etc/httpd/conf/httpd.conf file points to /etc/httpd/run as the folder for the .pid file and there is no such file there. So if that’s where VM is looking for the file I can see why it doesn’t find it. BUT - how has this taken place, what can be done and how can I prevent it from happening again?

Dave

The place where Webmin looks for the PID file is configured in Webmin -> Servers -> Apache Webserver -> Module Config -> System Configuration.

The place where Apache stores its PID file is configured in Webmin -> Servers -> Apache Webserver -> Global Configuration -> Miscellaneous.

The corresponding config file entry for me reads:

#
# PidFile: The file in which the server should record its process
# identification number when it starts.
# This needs to be set in /etc/apache2/envvars
#
PidFile ${APACHE_PID_FILE}

Might be different for you of course. And how they got out of sync for you, if they did, I can’t tell right off the bat. :slight_smile:

But. If instead of running those commands I go to Webmin | Servers | Apache Webserver and then click on Stop Apache (in the top right corner) then wait for the screen to refresh and click on Start Apache that DOES work. I find this a) handy because it works b) disconcerting because it seems to me to imply that the actions are not equivalent, which I would expect them to be. So what is the difference in what the scripts do in the two cases? I found this exact behaviour in an instance of VM GPl on another machine, too.

Bug?

Dave

I see from the httpd.conf file that the .pid file is supposed to be in the /etc/httpd/run folder. On one of my VM instances this file is present and correct. On the other it isn’t (it’s absent). That’s the one that gives the erroneous system information display. The server is running and serving pages but the inaccuracy in the display is troubling. Also there is some (related?) issue that periodically results in all but one instance of the daemon being killed off. This gives extremely slow response and makes for grumpy users. Ideas?

Dave

I am also having this issue on 3 VMPro and 1 VMGPL CentOS 5.5 installs . Three of them have Coldfusion running along side and one is running railo with tomcat which I though may have something to do with it. I have changed the PIDFILE path in httpd.conf to match the init script path but it didn’t fix the problem.

At some point after a few days the PID file disappears and the red X appears in the panel. Although apache is still working it looks like the main process which runs as root is missing but all the children are still there. I have to kill the processes to start apache again with the init script. Pid file will appear in the right place.

I am not sure if it happens at this time but I did find this in the logs every week at the exact same time:

/var/log/cron.1:Mar 20 04:02:01 myhostname crond[26715]: (root) CMD (run-parts /etc/cron.daily) /var/log/cron.1:Mar 20 04:02:01 myhostname crond[26716]: (root) CMD (/etc/webmin/virtual-server/collectinfo.pl) /var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] Digest: generating secret for digest authentication ... /var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] Digest: done /var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads. /var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [warn] No JkShmFile defined in httpd.conf. Using default /etc/httpd/logs/jk-runtime-status /var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations /var/log/httpd/mod_jk.log:[Sun Mar 20 04:02:01.922 2011] [21311:654213888] [info] init_jk::mod_jk.c (3183): mod_jk/1.2.28 initialized /var/log/httpd/error_log.1:[Sun Mar 20 04:02:01 2011] [notice] SIGHUP received. Attempting to restart /var/log/messages:Mar 20 04:02:01 myhostname syslogd 1.4.1: restart.

/var/log/httpd/error_log.2:[Thu Mar 17 04:02:02 2011] [notice] SIGHUP received. Attempting to restart
/var/log/httpd/error_log.1:[Sun Mar 20 04:02:01 2011] [notice] SIGHUP received. Attempting to restart
/var/log/httpd/error_log.4:[Sun Feb 20 04:02:01 2011] [notice] SIGHUP received. Attempting to restart

I just changed Webmin -> Servers -> Apache Webserver -> Module Config -> System Configuration. to point directly to the file instead of auto detect which was set by default. Lets hope this will fix it.

Any suggestions on how I can track down what kills the main httpd process if this doesn’t solve the problem?

Thanks,

Stefan

The changes didn’t fix the problem. It happened again at 4am, here are the logs from today:

[root@myhostname:~] $ grep -r “Mar 27 04:02:0” /var/log/
/var/log/cron.1:Mar 27 04:02:01 myhostname crond[9153]: (root) CMD (run-parts /etc/cron.daily)
/var/log/cron.1:Mar 27 04:02:01 myhostname crond[9156]: (root) CMD (/etc/webmin/virtual-server/collectinfo.pl)
/var/log/httpd/error_log.1:[Sun Mar 27 04:02:01 2011] [notice] SIGHUP received. Attempting to restart
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] Digest: generating secret for digest authentication …
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] Digest: done
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads.
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [warn] No JkShmFile defined in httpd.conf. Using default /etc/httpd/logs/jk-runtime-status
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] Apache/2.2.3 (CentOS) configured – resuming normal operations
/var/log/httpd/mod_jk.log:[Sun Mar 27 04:02:01.932 2011] [15766:143188736] [info] init_jk::mod_jk.c (3183): mod_jk/1.2.28 initialized
/var/log/messages:Mar 27 04:02:01 myhostname syslogd 1.4.1: restart.

Virtualmin version 3.83 Pro Webmin version 1.530 Kernel and CPU Linux 2.6.18-194.26.1.el5xen on x86_64 Operating system CentOS Linux 5.5 Virtual memory 511.99 MB total, 384 kB used Real memory 1024 MB total, 555.35 MB used Local disk space 34.46 GB total, 4.34 GB used

Any suggestions where to look next?

Stefan