Discussion:
munin-node problems with plugin output
Harnau, Sebastian
2010-01-13 16:46:30 UTC
Permalink
Hello,

i tried to build munin 1.4.3 for SLES 10 / 11 manually from sources and
i thought it went fine:
munin server behaves as it should behave, but munin-node seems to be
broken.

I can run the plugins via munin-run without any problems:

lx1016:/usr/lib/munin/plugins # munin-run cpu
user.value 65941756
nice.value 101256810
system.value 15934643
idle.value 385659485
iowait.value 42223921
irq.value 174098
softirq.value 641320
steal.value 0

But when i try to communicate via telnet with munin-node it doesn't
work:

lx1016:/usr/lib/munin/plugins # telnet localhost 4949
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at lx1016.iv.local
list
acpi bonding_err_bond0 cpu df df_inode entropy forks fw_packets
http_loadtime if_bond0 if_err_bond0 if_err_eth0 if_err_eth1 if_eth0
if_eth1 interrupts iostat iostat_ios irqstats linux_diskstat_iops_sda
linux_diskstat_iops_sdb linux_diskstat_latency_sda
linux_diskstat_latency_sdb linux_diskstat_throughput_sda
linux_diskstat_throughput_sdb load memory munin_stats netstat nfsd nfsd4
open_files open_inodes postfix_mailqueue postfix_mailvolume proc_pri
processes swap threads uptime users vmstat
nodes
lx1016.iv.local
.
fetch cpu
#.

There isn't any output for the listed plugins.

I followed the instructions in the wiki and activated debugmode for the
node. Then i found in the node's logfile:

2010/01/13-17:30:04 [5287] DEBUG: < fetch cpu
2010/01/13-17:30:04 [5287] DEBUG: Running command "fetch cpu".
2010/01/13-17:30:04 [5287] Error output from cpu:
2010/01/13-17:30:04 [5287] # Set /rgid/ruid/egid/euid/ to
/107/65534/107 107 /65534/
2010/01/13-17:30:04 [5287] # Setting up environment
2010/01/13-17:30:04 [5287] # About to run '/etc/munin/plugins/cpu'

I haven't found the part in the source which produces the wrong output
so I'm not able to unterstand why running the plugins over telnet -->
munin-node has other output than running plugins via munin-run or
directly via shell:

lx1016:/usr/lib/munin/plugins # /etc/munin/plugins/cpu
user.value 65949345
nice.value 101267226
system.value 15937120
idle.value 385712241
iowait.value 42225806
irq.value 174114
softirq.value 641389
steal.value 0

I would be happy if someone could give me a hint where to find the
problem in my scenario...

Thanks a lot, Sebastian.


_____________________________________________________________________

IVI Informationsverarbeitungs GmbH
Itzehoer Platz, 25524 Itzehoe
Geschäftsführer: Uwe Müller, Stefan Schwalbach.
Sitz: Itzehoe, Registergericht: Amtsgericht Pinneberg
HRB 2073 IZ, Ust.-ID-Nr. DE 134 777 598
_____________________________________________________________________
Nicolai Langfeldt
2010-01-13 22:35:35 UTC
Permalink
Post by Harnau, Sebastian
i tried to build munin 1.4.3 for SLES 10 / 11 manually from sources
...
Post by Harnau, Sebastian
lx1016:/usr/lib/munin/plugins # munin-run cpu
user.value 65941756
nice.value 101256810
system.value 15934643
idle.value 385659485
iowait.value 42223921
irq.value 174098
softirq.value 641320
steal.value 0
...
Post by Harnau, Sebastian
# munin node at lx1016.iv.local
list
acpi bonding_err_bond0 cpu df df_inode entropy forks fw_packets
...
Post by Harnau, Sebastian
.
fetch cpu
#.
There isn't any output for the listed plugins.
I can't even start to imagine how this happens. Are you running selinux or apparmour? If so please try to disable it entirely.

The only difference between munin-node and munin-run that I can think of is that munin-node collects all the plugin output (STDOUT _and_ STDERR) in a buffer before sending it onto the network, while munin-run does not do any redirection tricks.
Post by Harnau, Sebastian
I haven't found the part in the source which produces the wrong output
so I'm not able to unterstand why running the plugins over telnet -->
munin-node has other output than running plugins via munin-run or
...
Post by Harnau, Sebastian
I would be happy if someone could give me a hint where to find the
problem in my scenario...
Is it possible to give me remote access to the system where this happens?

Nicolai
Aleksander Kamenik
2010-01-13 22:38:22 UTC
Permalink
Post by Harnau, Sebastian
Hello,
i tried to build munin 1.4.3 for SLES 10 / 11 manually from sources and
munin server behaves as it should behave, but munin-node seems to be broken.
I'm successfully running several version 1.4.3 munin-nodes on SLES10SP2.
Random ideas:

Check the audit logs from apparmor (/var/log/audit/audit.log).

Did you install Net::Server, Net::Server::Fork and Time::HiRes from
CPAN? (I did)

Regards,
--
Aleksander Kamenik
System Administrator
Krediidiinfo AS
an Experian Company
Phone: +372 665 9649
Email: ***@krediidiinfo.ee

http://www.krediidiinfo.ee/
http://www.experiangroup.com/
Harnau, Sebastian
2010-01-14 08:54:07 UTC
Permalink
Post by Aleksander Kamenik
Post by Harnau, Sebastian
Hello,
i tried to build munin 1.4.3 for SLES 10 / 11 manually from sources
munin server behaves as it should behave, but munin-node seems to be broken.
I'm successfully running several version 1.4.3 munin-nodes on
SLES10SP2.
Post by Aleksander Kamenik
Check the audit logs from apparmor (/var/log/audit/audit.log).
No evidence leading to my problem...
Post by Aleksander Kamenik
Did you install Net::Server, Net::Server::Fork and Time::HiRes from
CPAN? (I did)

I tried this and after rebuilding packages, munin-node seems to work.
Thanks for your idea... :-)
Cu, Sebastian.

_____________________________________________________________________

IVI Informationsverarbeitungs GmbH
Itzehoer Platz, 25524 Itzehoe
Geschäftsführer: Uwe Müller, Stefan Schwalbach.
Sitz: Itzehoe, Registergericht: Amtsgericht Pinneberg
HRB 2073 IZ, Ust.-ID-Nr. DE 134 777 598
_____________________________________________________________________
Loading...