Error! MariaDB is not running on your system - database list could not be retrieved

SYSTEM INFORMATION
OS type and version CentOS Linux 7.9.2009
Webmin version 1.984
Virtualmin version 6.17
Related packages SUGGESTED

Hi guys. I often get this message. Any thoughts?

Error! MariaDB is not running on your system - database list could not be retrieved.
MariaDB error message: The full MariaDB error message was : DBI connect failed : Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’ (111)

MySQL Error log:

220527 17:54:24 [Warning] IP address '205.210.31.129' could not be resolved: Name or service not known

220527 18:53:45 mysqld_safe Number of processes running now: 0

220527 18:53:45 mysqld_safe mysqld restarted

220527 18:53:45 [Note] /usr/libexec/mysqld (mysqld 5.5.68-MariaDB) starting as process 20724 ...

220527 18:53:45 InnoDB: The InnoDB memory heap is disabled

220527 18:53:45 InnoDB: Mutexes and rw_locks use GCC atomic builtins

220527 18:53:45 InnoDB: Compressed tables use zlib 1.2.7

220527 18:53:45 InnoDB: Using Linux native AIO

220527 18:53:45 InnoDB: Initializing buffer pool, size = 128.0M

InnoDB: mmap(137756672 bytes) failed; errno 12

220527 18:53:45 InnoDB: Completed initialization of buffer pool

220527 18:53:45 InnoDB: Fatal error: cannot allocate memory for the buffer pool

220527 18:53:45 [ERROR] Plugin 'InnoDB' init function returned error.

220527 18:53:45 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

220527 18:53:45 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)

220527 18:53:46 [Note] Plugin 'FEEDBACK' is disabled.

220527 18:53:46 [ERROR] Unknown/unsupported storage engine: InnoDB

220527 18:53:46 [ERROR] Aborting

220527 18:53:46 [Note] /usr/libexec/mysqld: Shutdown complete

220527 18:53:46 mysqld_safe mysqld from pid file /var/run/mariadb/mariadb.pid ended

220528 22:24:21 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

220528 22:24:21 [Note] /usr/libexec/mysqld (mysqld 5.5.68-MariaDB) starting as process 16394 ...

220528 22:24:21 InnoDB: The InnoDB memory heap is disabled

220528 22:24:21 InnoDB: Mutexes and rw_locks use GCC atomic builtins

220528 22:24:21 InnoDB: Compressed tables use zlib 1.2.7

220528 22:24:21 InnoDB: Using Linux native AIO

220528 22:24:21 InnoDB: Initializing buffer pool, size = 128.0M

220528 22:24:21 InnoDB: Completed initialization of buffer pool

220528 22:24:21 InnoDB: highest supported file format is Barracuda.

220528 22:24:21 InnoDB: Starting crash recovery from checkpoint LSN=372208719

InnoDB: Restoring possible half-written data pages from the doublewrite buffer...

220528 22:24:21 InnoDB: Starting final batch to recover 1 pages from redo log

220528 22:24:22 InnoDB: Waiting for the background threads to start

220528 22:24:23 Percona XtraDB (http://www.percona.com) 5.5.61-MariaDB-38.13 started; log sequence number 372212044

220528 22:24:23 [Note] Plugin 'FEEDBACK' is disabled.

220528 22:24:23 [Note] Server socket created on IP: '0.0.0.0'.

220528 22:24:23 [Note] Event Scheduler: Loaded 0 events

220528 22:24:23 [Note] /usr/libexec/mysqld: ready for connections.

Version: '5.5.68-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server

220528 22:24:28 [ERROR] mysqld: Table './ibilling/sys_invoiceitems' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './ibilling/sys_invoiceitems'

220528 22:24:28 [ERROR] mysqld: Table './ibilling/sys_invoices' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './ibilling/sys_invoices'

220528 22:24:28 [ERROR] mysqld: Table './ibilling/sys_logs' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './ibilling/sys_logs'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_client_custom' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_client_custom'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_clients' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_clients'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_invoice_amounts' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_invoice_amounts'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_invoice_groups' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_invoice_groups'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_invoice_item_amounts' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_invoice_item_amounts'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_invoice_items' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_invoice_items'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_invoices' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_invoices'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_payments' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_payments'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_quote_amounts' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_quote_amounts'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_quote_item_amounts' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_quote_item_amounts'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_quote_items' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_quote_items'

220528 22:24:28 [ERROR] mysqld: Table './nkp_pisha/osssb_quotes' is marked as crashed and should be repaired

220528 22:24:28 [Warning] Checking table: './nkp_pisha/osssb_quotes'

============================================

/etc/my.cnf

[mysqld]

datadir=/var/lib/mysql

socket=/var/lib/mysql/mysql.sock

symbolic-links=0

innodb_file_per_table = 1

[mysqld_safe]

log-error=/var/log/mariadb/mariadb.log

pid-file=/var/run/mariadb/mariadb.pid

!includedir /etc/my.cnf.d

220527 18:53:45 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)
220527 18:53:46 [Note] Plugin ‘FEEDBACK’ is disabled.

That suggest it a issue with available ram.

Steve

I have a VPS with 4 vCPU Cores and 8 GB RAM.

Have you tried increasing buffer size?

Not yet. I will increase the buffer size and post the results here. Thank you!

I set the innodb_buffer_pool_size = 1600M

Once you address the memory issue, you also have crashed tables. You’ll need to repair them (as suggested by the log).

I run the mysqltuner on my VPS and followed the recommendations. Till now everything is working perfect. Will see

New MySQL server configuration:

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

innodb_stats_on_metadata = OFF
performance_schema = on
symbolic-links = 0
skip-name-resolve = 1
innodb_file_per_table = 1
innodb_buffer_pool_size = 1600M
innodb_log_file_size = 200M
join_buffer_size = 160M
tmp_table_size = 24M
max_heap_table_size = 24M
key_buffer_size = 25M
thread_cache_size = 6

[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid


!includedir /etc/my.cnf.d

I think the error is not related to configuration. I think your system is actually running out of RAM. I don’t know why that would be, as 8GB is plenty for most small scale web hosting needs. But…well, the error indicates it tried to allocate the 128MB and failed. I do not think increasing it (by a huge amount) will help here. I think it will make it more likely to fail to allocate that much memory.

1 Like

I set for some Apllications this on 256MB , if testing i think is always better to do that in smaller steps and see if problems are less or later. While if a Programm BUG and you set the value much to high, you still get a suprise but later. ( is my opinion about this value) ( if starting in some programms migrations then i needed more in the past then the 256MB, but migrations you do once and a while so then only)

If it’s me, I’d simply rerun the Virtualmin post install configuration and set it at defaults, reboot and go from there.

If I re-run the virtualmin post install configuration and set to defaults values something will be erased?

Nothing important will be erased. If only modifies configuration values, it won’t erase any data or mess with your sites.

But, I’m not confident that’s going to do anything useful here. I’d like to see evidence the database is where all your memory is going. Could be something else entirely. Check top or htop for clues.

Today another error:

220601 00:02:38 mysqld_safe mysqld restarted
220601 0:02:43 [Note] /usr/libexec/mysqld (mysqld 5.5.68-MariaDB) starting as process 5376 …
220601 0:02:43 InnoDB: The InnoDB memory heap is disabled
220601 0:02:43 InnoDB: Mutexes and rw_locks use GCC atomic builtins
220601 0:02:43 InnoDB: Compressed tables use zlib 1.2.7
220601 0:02:43 InnoDB: Using Linux native AIO
220601 0:02:43 InnoDB: Initializing buffer pool, size = 1.6G
InnoDB: mmap(1721958400 bytes) failed; errno 12
220601 0:02:43 InnoDB: Completed initialization of buffer pool
220601 0:02:43 InnoDB: Fatal error: cannot allocate memory for the buffer pool
220601 0:02:43 [ERROR] Plugin ‘InnoDB’ init function returned error.
220601 0:02:43 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
220601 0:02:44 [Note] Plugin ‘FEEDBACK’ is disabled.
220601 0:02:44 [ERROR] Unknown/unsupported storage engine: InnoDB
220601 0:02:44 [ERROR] Aborting

Look, I’m telling you you need to figure out what’s consuming all your memory. To repeat: “I’d like to see evidence the database is where all your memory is going. Could be something else entirely. Check top or htop for clues.”

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 
2546 root      20   0 3467232  98292  64664 S   1.7  1.2   2297:43 fail2ban-server
 4343 root      20   0  277296  58164    436 S   0.3  0.7  29:33.39 /usr/bin/spamd
11608 mysql     20   0 3043240 622456   9340 S   0.3  7.8   0:50.67 mysqld
15305 apache    20   0  255036   6584   2788 S   0.3  0.1   0:00.08 httpd
17033 root      20   0   95328  18176   1068 S   0.3  0.2   0:30.29 perl
19210 root      20   0  162504   2700   1580 R   0.3  0.0   0:00.04 top
24573 nkp-pis+  20   0  389148  19076  11540 S   0.3  0.2   0:00.89 php-cgi
    1 root      20   0  199252   3116   1632 S   0.0  0.0  98:33.82 systemd
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.79 kthreadd
    4 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
    6 root      20   0       0      0      0 S   0.0  0.0   3:54.91 ksoftirqd/0
    7 root      rt   0       0      0      0 S   0.0  0.0   2:24.09 migration/0
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
    9 root      20   0       0      0      0 S   0.0  0.0 179:11.87 rcu_sched
   10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-drain
   11 root      rt   0       0      0      0 S   0.0  0.0   0:56.36 watchdog/0
   12 root      rt   0       0      0      0 S   0.0  0.0   0:47.48 watchdog/1
   13 root      rt   0       0      0      0 S   0.0  0.0   2:12.52 migration/1
   14 root      20   0       0      0      0 S   0.0  0.0   4:22.13 ksoftirqd/1
   16 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/1:0H
   17 root      rt   0       0      0      0 S   0.0  0.0   0:33.82 watchdog/2
   18 root      rt   0       0      0      0 S   0.0  0.0   2:29.33 migration/2
   19 root      20   0       0      0      0 S   0.0  0.0   3:41.25 ksoftirqd/2
   21 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/2:0H
   22 root      rt   0       0      0      0 S   0.0  0.0   1:09.06 watchdog/3
   23 root      rt   0       0      0      0 S   0.0  0.0   2:16.07 migration/3
   24 root      20   0       0      0      0 S   0.0  0.0   4:30.29 ksoftirqd/3
   26 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/3:0H
   28 root      20   0       0      0      0 S   0.0  0.0   0:00.00 kdevtmpfs
   29 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 netns
   30 root      20   0       0      0      0 S   0.0  0.0   0:14.70 khungtaskd
   31 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 writeback
   32 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kintegrityd
   33 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 bioset
   34 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 bioset
   35 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 bioset
   36 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kblockd
   37 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 md
   38 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 edac-poller
   39 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 watchdogd
   46 root      20   0       0      0      0 S   0.0  0.0   0:50.51 kswapd0
   47 root      25   5       0      0      0 S   0.0  0.0   0:00.00 ksmd
cat /proc/meminfo
MemTotal:        8008632 kB
MemFree:         3551216 kB
MemAvailable:    3936012 kB
Buffers:           43832 kB
Cached:          1533160 kB
SwapCached:            0 kB
Active:          2615832 kB
Inactive:        1227092 kB
Active(anon):    2273160 kB
Inactive(anon):   980232 kB
Active(file):     342672 kB
Inactive(file):   246860 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:               136 kB
Writeback:             0 kB
AnonPages:       2265772 kB
Mapped:           541548 kB
Shmem:            987460 kB
Slab:             139380 kB
SReclaimable:      33524 kB
SUnreclaim:       105856 kB
KernelStack:        9600 kB
PageTables:       207596 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     4004316 kB
Committed_AS:   13530748 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       17012 kB
VmallocChunk:   34359718800 kB
Percpu:              832 kB
HardwareCorrupted:     0 kB
AnonHugePages:    321536 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      141160 kB
DirectMap2M:     4052992 kB
DirectMap1G:     6291456 kB
free -m
              total        used        free      shared  buff/cache   available
Mem:           7820        2809        3437         964        1573        3814
Swap:             0           0           0

That’s right.

Try these three options together:

innodb_buffer_pool_size = 512M
innodb_buffer_pool_instances = 4
innodb_buffer_pool_chunk_size = 64M

Increasing the pool size performed in chunks. I would suggest lowering the standard chunk size, alongside with raising pool instances. Most certainly, you must decrease pool size to 512M or even more down to 256M.


Why did you edit those default values in the first place? What made you do this? Did anything was failing to work?

1 Like

That would have helped (changed anything) only if you didn’t add many more of other options manually. We configure specific options upon wizard, but innodb_buffer_pool_size is not one of them, thus re-running the wizard wouldn’t change anything.

1 Like

I edited the defaults values cuz the first error was:
*220527 18:53:45 [ERROR] mysqld: Out of memory (Needed 128917504 bytes)*

I’ll try above options and post the results here. Thank you!