My MySQL server stopped and can't restart

SYSTEM INFORMATION
OS type and version Ubuntu Linux 20.04.2
Virtualmin version 6.16
Webmin version 1.973
VPS 2GB ram, 50GB memory

The virtualmin was working fine and suddenly I couldn’t connect to database.
When I try to start MySQL server I get the following error.

Job for mysql.service failed because the control process exited with error code. 
See "systemctl status mysql.service" and "journalctl -xe" for details.

Results for “systemctl status mysql.service”

● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2022-11-06 16:21:09 UTC; 7s ago
Process: 10731 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Process: 10739 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
Main PID: 10739 (code=exited, status=1/FAILURE)
Status: "Server shutdown complete"

Nov 06 16:21:08 dev systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Nov 06 16:21:08 dev systemd[1]: mysql.service: Failed with result 'exit-code'.
Nov 06 16:21:08 dev systemd[1]: Failed to start MySQL Community Server.
Nov 06 16:21:09 dev systemd[1]: MySQL.service: Scheduled restart job, restart counter is at 5.
Nov 06 16:21:09 dev systemd[1]: Stopped MySQL Community Server.
Nov 06 16:21:09 dev systemd[1]: mysql.service: Start request repeated too quickly.
Nov 06 16:21:09 dev systemd[1]: mysql.service: Failed with result 'exit-code'.
Nov 06 16:21:09 dev systemd[1]: Failed to start MySQL Community Server.

Results for “journalctl -xe”

● mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2022-11-06 16:21:09 UTC; 7s ago
Process: 10731 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Process: 10739 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
Main PID: 10739 (code=exited, status=1/FAILURE)
Status: "Server shutdown complete"

Nov 06 16:21:08 dev systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Nov 06 16:21:08 dev systemd[1]: mysql.service: Failed with result 'exit-code'.
Nov 06 16:21:08 dev systemd[1]: Failed to start MySQL Community Server.
Nov 06 16:21:09 dev systemd[1]: MySQL.service: Scheduled restart job, restart counter is at 5.
Nov 06 16:21:09 dev systemd[1]: Stopped MySQL Community Server.
Nov 06 16:21:09 dev systemd[1]: mysql.service: Start request repeated too quickly.
Nov 06 16:21:09 dev systemd[1]: mysql.service: Failed with result 'exit-code'.
Nov 06 16:21:09 dev systemd[1]: Failed to start MySQL Community Server.

- Journal -xe
    
     The job identifier is 24444 and the job result is done.
    Nov 06 16:22:07 dev systemd[1]: Starting MySQL Community Server...
    -- Subject: A start job for unit mysql.service has begun execution
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - A start job for unit mysql.service has begun execution.
    
    - The job identifier is 24444.
    Nov 06 16:22:09 dev systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
    -- Subject: Unit process exited
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - An ExecStart= process belonging to unit mysql.service has exited.
    
    - The process' exit code is 'exited' and its exit status is 1.
    Nov 06 16:22:09 dev systemd[1]: mysql.service: Failed with result 'exit-code'.
    -- Subject: Unit failed
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - The unit mysql.service has entered the 'failed' state with result 'exit-code'.
    Nov 06 16:22:09 dev systemd[1]: Failed to start MySQL Community Server.
    -- Subject: A start job for unit mysql.service has failed
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - A start job for unit mysql.service has finished with a failure.
    
    - The job identifier is 24444 and the job result is failed.
    Nov 06 16:22:09 dev systemd[1]: mysql.service: Scheduled restart job, restart counter is at 5.
    -- Subject: Automatic restarting of a unit has been scheduled
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - Automatic restarting of the unit mysql.service has been scheduled, as the result for
    -- the configured Restart= setting for the unit.
    Nov 06 16:22:09 dev systemd[1]: Stopped MySQL Community Server.
    -- Subject: A stop job for unit mysql.service has finished
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - A stop job for unit mysql.service has finished.
    
    - The job identifier is 24526 and the job result is done.
    Nov 06 16:22:09 dev systemd[1]: mysql.service: Start request repeated too quickly.
    Nov 06 16:22:09 dev systemd[1]: mysql.service: Failed with result 'exit-code'.
    -- Subject: Unit failed
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - The unit mysql.service has entered the 'failed' state with result 'exit-code'.
    Nov 06 16:22:09 dev systemd[1]: Failed to start MySQL Community Server.
    -- Subject: A start job for unit mysql.service has failed
    -- Defined-By: systemd
    -- Support: [http://www.ubuntu.com/support](http://www.ubuntu.com/support)
    
    - A start job for unit mysql.service has finished with a failure.
    
    - - The job identifier is 24526 and the job result is failed.
    Nov 06 16:22:11 dev named[780]: client @0x7fb2d809b9f0 86.19.33.185#647 (sl): query (cache) 'sl/ANY/IN' denied

Following is found in /etc/log/mysql/error.log

2022-11-06T18:04:02.526491Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31-0ubuntu0.20.04.1) starting as process 27882
2022-11-06T18:04:02.573393Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-11-06T18:04:03.350796Z 1 [ERROR] [MY-012526] [InnoDB] Upgrade is not supported after a crash or shutdown with innodb_fast_shutdown = 2. This redo log was created with MySQL 8.0.29, and it appears logically non empty. Please follow the instructions at http://dev.mysql.com/doc/refman/8.0/en/upgrading.html
2022-11-06T18:04:03.350857Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
2022-11-06T18:04:03.849798Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2022-11-06T18:04:03.850075Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2022-11-06T18:04:03.850104Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-11-06T18:04:03.850921Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.31-0ubuntu0.20.04.1)  (Ubuntu).
2022-11-06T18:04:04.891502Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31-0ubuntu0.20.04.1) starting as process 27933
2022-11-06T18:04:04.962733Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-11-06T18:04:05.855687Z 1 [ERROR] [MY-012526] [InnoDB] Upgrade is not supported after a crash or shutdown with innodb_fast_shutdown = 2. This redo log was created with MySQL 8.0.29, and it appears logically non empty. Please follow the instructions at http://dev.mysql.com/doc/refman/8.0/en/upgrading.html
2022-11-06T18:04:05.855734Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
2022-11-06T18:04:06.355552Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2022-11-06T18:04:06.355871Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2022-11-06T18:04:06.355904Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-11-06T18:04:06.356749Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.31-0ubuntu0.20.04.1)  (Ubuntu).
2022-11-06T18:04:07.397534Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31-0ubuntu0.20.04.1) starting as process 27971
2022-11-06T18:04:07.429282Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.

Kindly suggest me a lead on this. Spent whole day and couldn’t figure it out.

Thank you,
Regards

did you try removing innodb_fast_shutdown = 2 from config ?

Thank you for your reply!

I checked the MySQL config file (/etc/mysql/mysql.cnf). This is all I have in that file.

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

look for any .cnf files in /etc/mysql and also inside those 2 directories you see in my.cnf.

inside some .cnf file you might find the above parameter which you should try removing before restarting mysql.

Again, thank you very much for the reply.

I checked all the files under /etc/mysql and I couldn’t find any such entrance.

These are what I found.

file: my.cnf.fallback

!includedir /etc/mysql/conf.d/

File: debain.cnf

[client]
host     = localhost
user     = debian-sys-maint
password = rMT3qhIRrRyfCNC5
socket   = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host     = localhost
user     = debian-sys-maint
password = ##################
socket   = /var/run/mysqld/mysqld.sock

File: mysql.conf.d/mysql.cnf

[mysql]

File: mysql.conf.d/mysql.cnf

[mysqld]
user		= mysql
bind-address = 127.0.0.1
mysqlx-bind-address	= 127.0.0.1
key_buffer_size		= 16M
myisam-recover-options  = BACKUP
log_error = /var/log/mysql/error.log
max_binlog_size   = 100M
innodb_file_per_table = 1

There is nothing significant in the etc/mysql/conf.d folder.

I really wonder, is there any other way that I can recover my database? I only can find the .idb files in /var/lib/mysql. I couldn’t get the .frm.

Any help to restart the MySQL or recover my database would be appreciated. :slight_smile:

Thank you,
regards

try adding ```innodb_fast_shutdown = 0 in file :

and restarting.
maybe you have a full ram? do you have enough system resources (eg. ram) to run mysql?

p.s. don’t post passwords on public forums, even random ones like these…

Hello Dimitrist,

Thank you for the reminder. I have removed the password. :slightly_smiling_face:

A:
I have the VPS in digitalocean and just to make sure the ram is the issue, I upgraded it to 4Gb ram from 2Gb ram. Even after upgrading, the same error persists.

B:
I tried to add “innodb_fast_shutdown = 0” to /etc/mysql/mysql.conf.d/mysql.cnf. This is the error I got.

The MySQL client program
/usr/bin/mysql
does not appear to be the correct version. Webmin only supports MySQL versions 3 and above.
The command /usr/bin/mysql -V returned :

mysql: [ERROR] unknown variable 'innodb_fast_shutdown=0'.

C:
Then I tried to add the “innodb_fast_shutdown = 0” to /etc/mysql/mysql.cnf , this is the error I got.

mysql: [ERROR] Found option without preceding group in config file /etc/mysql/my.cnf at line 23.
mysql: [ERROR] Fatal error in defaults handling. Program aborted!

Any ideas on how to proceed?

Thank you,
Regards

no ideas from here. can help professionally (for a fee) , but can’t fix it from forum. i think mysql is not really supported anymore , only mariadb…

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.