How to check the memory usage of your embedded Linux system
After reading a forum post here I had to check the the memory usage off my OLinuXino too.
I have a bit more free men but also not that mush:
[root@alarm ~]# cat /proc/version
Linux version 3.7.4-dirty (chris@thinkpad) (gcc version 4.7.3 20121207 (release) [ARM/embedded-4_7-branch revision 194305] (GNU Tools for ARM Embedded3
[root@alarm ~]# cat /proc/meminfo |head -n 10
MemTotal: 52188 kB
MemFree: 17636 kB
Buffers: 2380 kB
Cached: 19232 kB
SwapCached: 0 kB
Active: 9036 kB
Inactive: 17256 kB
Active(anon): 4708 kB
Inactive(anon): 168 kB
Active(file): 4328 kB
[root@alarm ~]# ps -aux |head -n 10
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 2.1 4.8 4768 2516 ? Ss 01:00 0:03 /usr/lib/systemd/systemd
root 2 0.0 0.0 0 0 ? S 01:00 0:00 [kthreadd]
root 3 0.1 0.0 0 0 ? S 01:00 0:00 [ksoftirqd/0]
root 4 0.0 0.0 0 0 ? S 01:00 0:00 [kworker/0:0]
root 5 0.0 0.0 0 0 ? S< 01:00 0:00 [kworker/0:0H]
root 6 0.0 0.0 0 0 ? S 01:00 0:00 [kworker/u:0]
root 7 0.0 0.0 0 0 ? S< 01:00 0:00 [kworker/u:0H]
root 8 0.0 0.0 0 0 ? S 01:00 0:00 [watchdog/0]
root 9 0.0 0.0 0 0 ? S< 01:00 0:00 [khelper]
[root@alarm ~]#
Disable systemd-journald storage
It seems systemd-journald is using most memory on my system:
[root@alarm ~]# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5
10.5 0.4 69648 58 /usr/lib/systemd/systemd-journald
5.6 0.1 7040 96 /usr/sbin/syslog-ng -F
4.8 0.5 4768 1 /usr/lib/systemd/systemd
4.2 0.0 6272 94 /usr/sbin/sshd -D
3.3 0.1 3268 140 -bash
[root@alarm ~]#
Now we will tweak the memory usage of systemd-journald by setting the Storage-option to none, have a look here http://www.freedesktop.org/software/systemd/man/journald.conf.html.
[root@alarm ~]# vim /etc/systemd/journald.conf
[root@alarm ~]# cat /etc/systemd/journald.conf
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
#
# See journald.conf(5) for details
[Journal]
Storage=none
#Compress=yes
#Seal=yes
#SplitMode=login
#RateLimitInterval=10s
#RateLimitBurst=200
#SystemMaxUse=
#SystemKeepFree=
#SystemMaxFileSize=
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#MaxRetentionSec=
#MaxFileSec=1month
#ForwardToSyslog=yes
#ForwardToKMsg=no
#ForwardToConsole=no
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
[root@alarm ~]# reboot
A little bit better, now syslog-ng uses the most memory:
[root@alarm ~]# cat /proc/meminfo |head -n 10
MemTotal: 52188 kB
MemFree: 19732 kB
Buffers: 2332 kB
Cached: 17728 kB
SwapCached: 0 kB
Active: 8844 kB
Inactive: 15744 kB
Active(anon): 4556 kB
Inactive(anon): 168 kB
Active(file): 4288 kB
[root@alarm ~]# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5
5.6 0.8 7040 89 /usr/sbin/syslog-ng -F
4.8 2.9 4768 1 /usr/lib/systemd/systemd
4.2 0.1 6272 88 /usr/sbin/sshd -D
3.2 0.8 3880 124 login -- root
3.2 0.7 3152 141 -bash
[root@alarm ~]#
If you don’t need /var/log/messages then you can stop syslog-ng:
[root@alarm ~]# systemctl disable syslog-ng.service
[root@alarm ~]# systemctl stop syslog-ng.service
[root@alarm ~]# reboot
But it is running again???:
[root@alarm ~]# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5
5.6 4.1 7040 90 /usr/sbin/syslog-ng -F
4.8 10.5 4768 1 /usr/lib/systemd/systemd
4.2 0.9 6272 89 /usr/sbin/sshd -D
3.2 4.7 3152 140 -bash
3.2 4.6 3880 122 login -- root
[root@alarm ~]# cat /proc/meminfo |head -n 10
MemTotal: 52188 kB
MemFree: 19536 kB
Buffers: 2324 kB
Cached: 17952 kB
SwapCached: 0 kB
Active: 8872 kB
Inactive: 15908 kB
Active(anon): 4532 kB
Inactive(anon): 168 kB
Active(file): 4340 kB
[root@alarm ~]# systemctl status syslog-ng.service
syslog-ng.service - System Logger Daemon
Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; disabled)
Active: active (running) since Thu 1970-01-01 01:29:11 CET; 2min 12s ago
Docs: man:syslog-ng(8)
Main PID: 90 (syslog-ng)
CGroup: name=systemd:/system/syslog-ng.service
└───90 /usr/sbin/syslog-ng -F
[root@alarm ~]#
Now I’m pissed off.
Make sure that systemd is your default init system:
[root@alarm ~]# cat /proc/1/comm
systemd
[root@alarm ~]# cat /proc/cmdline
noinitrd console=ttyAMA0,115200 root=/dev/mmcblk0p2 init=/usr/lib/systemd/systemd rw rootwait ssp1=mmc
[root@alarm ~]#
Then remove the old stuff:
[root@alarm ~]# pacman -Sy systemd systemd-sysvcompat
:: Synchronizing package databases...
core 41.1 KiB 6.52K/s 00:06 [######################] 100%
error: failed retrieving file 'extra.db' from mirror.archlinuxarm.org : Operation too slow. Less than 1024 bytes/sec transferred the last 10 seconds
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from mirror.archlinuxarm.org : Operation too slow. Less than 1024 bytes/sec transferred the last 10 secons
error: failed to update community (download library error)
alarm 5.1 KiB 5.72K/s 00:01 [######################] 100%
aur 13.9 KiB 1471B/s 00:10 [######################] 100%
warning: systemd-197-4 is up to date -- reinstalling
resolving dependencies...
looking for inter-conflicts...
:: systemd-sysvcompat and sysvinit are in conflict. Remove sysvinit? [y/N] y
Targets (3): sysvinit-2.88-9 [removal] systemd-197-4 systemd-sysvcompat-197-4
Total Download Size: 0.01 MiB
Total Installed Size: 11.95 MiB
Net Upgrade Size: -0.09 MiB
Proceed with installation? [Y/n] y
:: Retrieving packages from core...
systemd-sysvcompat-... 5.3 KiB 103K/s 00:00 [######################] 100%
(2/2) checking package integrity [######################] 100%
(2/2) loading package files [######################] 100%
(2/2) checking for file conflicts [######################] 100%
(3/3) checking available disk space [######################] 100%
(1/1) removing sysvinit [######################] 100%
(1/2) upgrading systemd [######################] 100%
==> Warning: setcap failed, falling back to setuid root on /usr/bin/systemd-detect-virt
(2/2) installing systemd-sysvcompat [######################] 100%
[root@alarm ~]# pacman -Rs syslog-ng initscripts sysvinit
error: target not found: sysvinit
[root@alarm ~]# pacman -Rs syslog-ng initscripts
checking dependencies...
Targets (3): eventlog-0.2.12-3 initscripts-2012.10.1-1 syslog-ng-3.3.7-1
Total Removed Size: 1.97 MiB
Do you want to remove these packages? [Y/n] y
(1/3) removing initscripts [######################] 100%
warning: /etc/rc.local saved as /etc/rc.local.pacsave
warning: /etc/rc.conf saved as /etc/rc.conf.pacsave
warning: /etc/inittab saved as /etc/inittab.pacsave
(2/3) removing syslog-ng [######################] 100%
(3/3) removing eventlog [######################] 100%
[root@alarm ~]# reboot
Now we have more free mem:
[root@alarm ~]# cat /proc/meminfo |head -n 10
MemTotal: 52188 kB
MemFree: 25116 kB
Buffers: 2220 kB
Cached: 14008 kB
SwapCached: 0 kB
Active: 7360 kB
Inactive: 11768 kB
Active(anon): 2928 kB
Inactive(anon): 48 kB
Active(file): 4432 kB
[root@alarm ~]# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -10
4.6 0.4 4600 1 /usr/lib/systemd/systemd
3.2 0.1 3880 101 login -- root
3.2 0.0 3268 113 -bash
3.1 0.0 5416 89 /usr/bin/ntpd -g -u ntp:ntp
2.4 0.1 8552 48 /usr/lib/systemd/systemd-udevd
2.4 0.0 3108 83 /usr/lib/systemd/systemd-logind
2.4 0.0 2692 84 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
2.0 0.1 3460 51 /usr/lib/systemd/systemd-journald
2.0 0.0 5416 98 /usr/bin/ntpd -g -u ntp:ntp
1.5 0.0 2556 150 ps -eo pmem,pcpu,vsize,pid,cmd
[root@alarm ~]# free
total used free shared buffers cached
Mem: 52188 27016 25172 0 2248 14024
-/+ buffers/cache: 10744 41444
Swap: 0 0 0
[root@alarm ~]#
Arch Linux ARM with stock kernel
Newest Arch Linux ARM from http://archlinuxarm.org/platforms/armv5/olinuxino with stock kernel:
[root@olinuxino ~]# cat /proc/version
Linux version 3.7.2-2-ARCH (nobody@panda2) (gcc version 4.7.2 (GCC) ) #1 PREEMPT Thu Jan 17 07:51:19 UTC 2013
[root@olinuxino ~]# cat /proc/meminfo |head -n 10
MemTotal: 52436 kB
MemFree: 10368 kB
Buffers: 6996 kB
Cached: 20988 kB
SwapCached: 0 kB
Active: 12800 kB
Inactive: 20036 kB
Active(anon): 4864 kB
Inactive(anon): 88 kB
Active(file): 7936 kB
[root@olinuxino ~]# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -10
9.9 0.3 38972 617 /usr/lib/systemd/systemd-journald
5.6 0.1 7040 977 /usr/sbin/syslog-ng -F
4.7 0.3 4600 1 /sbin/init
4.2 0.0 6272 974 /usr/sbin/sshd -D
3.6 0.1 5332 1488 -bash
3.3 0.0 5912 1029 login -- root
2.5 0.3 9112 614 /usr/lib/systemd/systemd-udevd
2.4 0.0 3108 975 /usr/lib/systemd/systemd-logind
2.4 0.0 2692 976 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
2.3 40.9 4804 973 /usr/sbin/crond -n
[root@olinuxino ~]#