Pandora FMS Community

Help! All my plugin server modules have stopped working after a failure at AWS

JonRussell · 13 ene 2019, 23:52 · 17 mensajes · 107 visitas

  1. Inicio del debate

    Hi, So, AWS had a network failure over the weekend, and I got an email saying my VMs were affected. They were all frozen and one was off completely (the Pandora server was the VM that was off ! :-( ) I rebooted them all, but Pandora seems to be playing up. I have two issues, which might be related ?

    • On boot, Pandora Server wont start. but if I start it a few minutes later, it seems to start ok?
    • All my plugin server modules have stopped. Pandora is still receiving data from all the agents and logging / graphing it ok. But all the server side plugins say "Unknown" "20 hours"

    Any ideas ?

    
    [root@pandora centos]# cat /var/log/pandora/pandora_server.log
     
    2019-01-13 17:34:21 pandora [V1] Launching 7.0NG.730 181217
    2019-01-13 17:34:21 pandora [V1] Config options: Logfile at /var/log/pandora/pandora_server.log, Basepath is /etc/pandora/pandora_server.conf, Checksum is 0, Master is 1, SNMP Console is 0, Server Threshold at 5 sec, verbosity at 3, Alert Threshold at 60, ServerName is 'pandora'
    2019-01-13 17:34:21 pandora [V1] Pandora FMS Enterprise module not available.
    2019-01-13 17:34:21 pandora [V1] [E] 'pandora': DBI connect('pandora:127.0.0.1:3306','pandora',...) failed: Can't connect to MySQL server on '127.0.0.1' (111) at /usr/lib/perl5/PandoraFMS/DB.pm line 125.
    2019-01-13 17:34:21 pandora [V1]  Server 'pandora' unhandled error.
     
    

     

    
    [root@pandora centos]# cat /var/log/pandora/pandora_server.error
     
    2019-01-13 17:34:21 - pandora Starting Pandora FMS Server. Error logging activated.
    Use of uninitialized value $Config{"rb_product_name"} in concatenation (.) or string at /usr/bin/pandora_server line 215.
    DBI connect('pandora:127.0.0.1:3306','pandora',...) failed: Can't connect to MySQL server on '127.0.0.1' (111) at /usr/lib/perl5/PandoraFMS/DB.pm line 125.
     
    
    
    [root@pandora centos]# systemctl status pandora_server                                        
    ● pandora_server.service - LSB: Pandora FMS Server startup script
       Loaded: loaded (/etc/rc.d/init.d/pandora_server; bad; vendor preset: disabled)
       Active: failed (Result: exit-code) since Sun 2019-01-13 17:34:22 UTC; 2h 20min ago
         Docs: man:systemd-sysv-generator(8)
      Process: 3012 ExecStart=/etc/rc.d/init.d/pandora_server start (code=exited, status=7)
    
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: Pandora FMS Server 7.0NG.730 Build 181217 Copyright (c) 2004-2018 Artica ST
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: This program is OpenSource, licensed under the terms of GPL License version 2.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: You can download latest versions and documentation at official web page.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]:  Backgrounding Pandora FMS Server process.
    
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Cannot start Pandora FMS Server. Aborted.
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Check Pandora FMS log files at '/var/log/pandora/pandora_server.error & pandora_s...r.log'
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service: control process exited, code=exited status=7
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Failed to start LSB: Pandora FMS Server startup script.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Unit pandora_server.service entered failed state.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service failed.
    

    mySQL *is* running. as the other agent data is being stored ? and when I run the plugins directly from the command line, they work fine. When the servers came up after the AWS network failure, I had some hostname issues. So, the hostnames have changed. I think I've corrected them all. Could that be related ? What part of Pandora runs the plugin scripts every 5 minutes ? Is it the Pandora Server ? How can I tell if it's attempting to run the scripts ? If Pandora server isn't running, would I still get the website and the agent data ? Thanks ! Jon.

  2. Hi, So, AWS had a network failure over the weekend, and I got an email saying my VMs were affected. They were all frozen and one was off completely (the Pandora server was the VM that was off ! :-( ) I rebooted them all, but Pandora seems to be playing up. I have two issues, which might be related ?

    • On boot, Pandora Server wont start. but if I start it a few minutes later, it seems to start ok?
    • All my plugin server modules have stopped. Pandora is still receiving data from all the agents and logging / graphing it ok. But all the server side plugins say "Unknown" "20 hours"

    Any ideas ?

    
    [root@pandora centos]# cat /var/log/pandora/pandora_server.log
     
    2019-01-13 17:34:21 pandora [V1] Launching 7.0NG.730 181217
    2019-01-13 17:34:21 pandora [V1] Config options: Logfile at /var/log/pandora/pandora_server.log, Basepath is /etc/pandora/pandora_server.conf, Checksum is 0, Master is 1, SNMP Console is 0, Server Threshold at 5 sec, verbosity at 3, Alert Threshold at 60, ServerName is 'pandora'
    2019-01-13 17:34:21 pandora [V1] Pandora FMS Enterprise module not available.
    2019-01-13 17:34:21 pandora [V1] [E] 'pandora': DBI connect('pandora:127.0.0.1:3306','pandora',...) failed: Can't connect to MySQL server on '127.0.0.1' (111) at /usr/lib/perl5/PandoraFMS/DB.pm line 125.
    2019-01-13 17:34:21 pandora [V1]  Server 'pandora' unhandled error.
     
    

     

    
    [root@pandora centos]# cat /var/log/pandora/pandora_server.error
     
    2019-01-13 17:34:21 - pandora Starting Pandora FMS Server. Error logging activated.
    Use of uninitialized value $Config{"rb_product_name"} in concatenation (.) or string at /usr/bin/pandora_server line 215.
    DBI connect('pandora:127.0.0.1:3306','pandora',...) failed: Can't connect to MySQL server on '127.0.0.1' (111) at /usr/lib/perl5/PandoraFMS/DB.pm line 125.
     
    
    
    [root@pandora centos]# systemctl status pandora_server                                        
    ● pandora_server.service - LSB: Pandora FMS Server startup script
       Loaded: loaded (/etc/rc.d/init.d/pandora_server; bad; vendor preset: disabled)
       Active: failed (Result: exit-code) since Sun 2019-01-13 17:34:22 UTC; 2h 20min ago
         Docs: man:systemd-sysv-generator(8)
      Process: 3012 ExecStart=/etc/rc.d/init.d/pandora_server start (code=exited, status=7)
    
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: Pandora FMS Server 7.0NG.730 Build 181217 Copyright (c) 2004-2018 Artica ST
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: This program is OpenSource, licensed under the terms of GPL License version 2.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: You can download latest versions and documentation at official web page.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]:  Backgrounding Pandora FMS Server process.
    
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Cannot start Pandora FMS Server. Aborted.
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Check Pandora FMS log files at '/var/log/pandora/pandora_server.error & pandora_s...r.log'
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service: control process exited, code=exited status=7
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Failed to start LSB: Pandora FMS Server startup script.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Unit pandora_server.service entered failed state.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service failed.
    

    mySQL *is* running. as the other agent data is being stored ? and when I run the plugins directly from the command line, they work fine. When the servers came up after the AWS network failure, I had some hostname issues. So, the hostnames have changed. I think I've corrected them all. Could that be related ? What part of Pandora runs the plugin scripts every 5 minutes ? Is it the Pandora Server ? How can I tell if it's attempting to run the scripts ? If Pandora server isn't running, would I still get the website and the agent data ? Thanks ! Jon.

    Good morning JonRussell, 1) Can it be that Pandora FMS does not start on boot, because is trying to connect before MySQL is running? The error that Pandora FMS has is related with the pandora database. 2) The plugins can be done or by the agent or by the server, depending of the type of plugin used. If your case is the server plugins, it depends on the server. The interval of each script depends on the module/agent interval you have introduced. You can see in these modules if Pandora is doing, or not, this monitorization. Can you please attach me an screenshot of the error of the scripts of Unknown or 20 hours? Could it be that the information given at the script modules has changed because of the problems with the hostnames? 3) If Pandora server is not running, the website will appear, but all the data appered will be from the last time pandora server was up. Best regards, Alberto

  3. Hi Alberto, Thanks for replying. 1) Can it be that Pandora FMS does not start on boot, because is trying to connect before MySQL is running? The error that Pandora FMS has is related with the pandora database. Possibly, but I dont think I had this problem before the issues at the weekend. Is there a way to tell Pandora to wait for mySQL to start before trying to launch ? 2) The plugins can be done or by the agent or by the server, depending of the type of plugin used. If your case is the server plugins, it depends on the server. The interval of each script depends on the module/agent interval you have introduced. You can see in these modules if Pandora is doing, or not, this monitorization. Can you please attach me an screenshot of the error of the scripts of Unknown or 20 hours? Could it be that the information given at the script modules has changed because of the problems with the hostnames? These are server plugins. they are on the Pandora server in /usr/share/pandora_server/util/plugin/ I have three scripts I'm using,  sslcheck.sh webpage_content.sh webpage_speed.sh They check the SSL expiry, content of the web page and speed of the web page. I can run them all manually on the server :

    
    [root@pandora plugin]# ./sslcheck.sh www.teescottage.co.uk
    42
    [root@pandora plugin]# ./webpage_content.sh -u https://www.teescottage.co.uk/ -m Pumping Station
    1
    [root@pandora plugin]# ./webpage_speed.sh -t https://www.teescottage.co.uk
    0.297
    

    I have attached screenshots of one of the config pages and the Agent page. 3) If Pandora server is not running, the website will appear, but all the data appered will be from the last time pandora server was up. It seem to run if I manually start it. But "systemctl status pandora_server" still says "failed" ?

    
    [root@pandora plugin]# systemctl status pandora_server                     
    ● pandora_server.service - LSB: Pandora FMS Server startup script
       Loaded: loaded (/etc/rc.d/init.d/pandora_server; bad; vendor preset: disabled)
       Active: failed (Result: exit-code) since Sun 2019-01-13 17:34:22 UTC; 2 days ago
         Docs: man:systemd-sysv-generator(8)
      Process: 3012 ExecStart=/etc/rc.d/init.d/pandora_server start (code=exited, status=7)
    
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: Pandora FMS Server 7.0NG.730 Build 181217 Copyright (c) 2004-2018 Artica ST
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: This program is OpenSource, licensed under the terms of GPL License v...on 2.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: You can download latest versions and documentation at official web page.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]:
     Backgrounding Pandora FMS Server process.
    
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Cannot start Pandora FMS Server. Aborted.
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Check Pandora FMS log files at '/var/log/pandora/pandora_server.error....log'
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service: control process exited, code=exited status=7
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Failed to start LSB: Pandora FMS Server startup script.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Unit pandora_server.service entered failed state.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service failed.
    

    Thanks Jon.

    [

    ](//pandorafms.com/community/media/forum/10841/27408=2618-Pandora-2.jpg)

    [

    ](//pandorafms.com/community/media/forum/10841/27408=2617-Pandora-1.jpg)

  4. Hi Alberto, Thanks for replying. 1) Can it be that Pandora FMS does not start on boot, because is trying to connect before MySQL is running? The error that Pandora FMS has is related with the pandora database. Possibly, but I dont think I had this problem before the issues at the weekend. Is there a way to tell Pandora to wait for mySQL to start before trying to launch? 2) The plugins can be done or by the agent or by the server, depending of the type of plugin used. If your case is the server plugins, it depends on the server. The interval of each script depends on the module/agent interval you have introduced. You can see in these modules if Pandora is doing, or not, this monitorization. Can you please attach me an screenshot of the error of the scripts of Unknown or 20 hours? Could it be that the information given at the script modules has changed because of the problems with the hostnames? These are server plugins. they are on the Pandora server in /usr/share/pandora_server/util/plugin/ I have three scripts I'm using,  sslcheck.sh webpage_content.sh webpage_speed.sh They check the SSL expiry, content of the web page and speed of the web page. I can run them all manually on the server :

    
    [root@pandora plugin]# ./sslcheck.sh www.teescottage.co.uk
    42
    [root@pandora plugin]# ./webpage_content.sh -u https://www.teescottage.co.uk/ -m Pumping Station
    1
    [root@pandora plugin]# ./webpage_speed.sh -t https://www.teescottage.co.uk
    0.297
    

    I have attached screenshots of one of the config pages and the Agent page. 3) If Pandora server is not running, the website will appear, but all the data appered will be from the last time pandora server was up. It seem to run if I manually start it. But "systemctl status pandora_server" still says "failed" ?

    
    [root@pandora plugin]# systemctl status pandora_server                     
    ● pandora_server.service - LSB: Pandora FMS Server startup script
       Loaded: loaded (/etc/rc.d/init.d/pandora_server; bad; vendor preset: disabled)
       Active: failed (Result: exit-code) since Sun 2019-01-13 17:34:22 UTC; 2 days ago
         Docs: man:systemd-sysv-generator(8)
      Process: 3012 ExecStart=/etc/rc.d/init.d/pandora_server start (code=exited, status=7)
    
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: Pandora FMS Server 7.0NG.730 Build 181217 Copyright (c) 2004-2018 Artica ST
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: This program is OpenSource, licensed under the terms of GPL License v...on 2.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]: You can download latest versions and documentation at official web page.
    Jan 13 17:34:21 pandora.red-herring.co.uk pandora_server[3012]:
     Backgrounding Pandora FMS Server process.
    
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Cannot start Pandora FMS Server. Aborted.
    Jan 13 17:34:22 pandora.red-herring.co.uk pandora_server[3012]: Check Pandora FMS log files at '/var/log/pandora/pandora_server.error....log'
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service: control process exited, code=exited status=7
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Failed to start LSB: Pandora FMS Server startup script.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: Unit pandora_server.service entered failed state.
    Jan 13 17:34:22 pandora.red-herring.co.uk systemd[1]: pandora_server.service failed.
    

    Thanks Jon.

    Good morning Jon, 1) You can try to use the command Chkconfig to add the mysql service to the list of services that starts on boot.  2) Can you please Force manually the module of any of the plugins to see if it works?¿ Can you revise the configuration of the plugin? Can you see if in /etc/pandora/pandora_server.conf the "pluginserver" is on 1? 3) Try to use /etc/init.d/status instead of systemctl. Sometimes i systemctl works bad. Alberto

    1. You can try to use the command Chkconfig to add the mysql service to the list of services that starts on boot.

     

    I'll try and add mySQL to this list, but mySQL *is* starting on boot. I just guess its taking time to start up. It is Pandora Server that is not starting on boot. It complains it cant connect to mySQL. But when I start it manually a few minutes later, it starts ok. I will try and add some dependency on mySQL to the Pandora startup parameters.

    [root@pandora plugin]# chkconfig
    
    netconsole              0:off   1:off   2:off   3:off   4:off   5:off   6:off
    network                 0:off   1:off   2:on    3:on    4:on    5:on    6:off
    pandora_agent_daemon    0:off   1:off   2:on    3:on    4:on    5:on    6:off
    pandora_server          0:off   1:off   2:on    3:on    4:on    5:on    6:off
    tentacle_serverd        0:off   1:off   2:on    3:on    4:on    5:on    6:off
    
    1. Can you please Force manually the module of any of the plugins to see if it works?¿ Can you revise the configuration of the plugin? Can you see if in /etc/pandora/pandora_server.conf the "pluginserver" is on 1?

    When I force the plugin, it gets stuck at the "refresh" icon for a long time and never executes. As I write this all three I clicked "force" on are still showing the circular arrow "refresh" icon, and its been almost 20 minutes ? I can edit the config and the changes update successfully. pluginserver is set to "1" in the pandora_server.conf. I just tried adding a new script in to the plugin directory. I registered it under "Servers" - "Plugins" and it created ok. I then added the server plugin to one of the agent pages under modules, and when I click "create" the new module is stuck blue in "not initialised" ?

    1. Try to use /etc/init.d/status instead of systemctl. Sometimes i systemctl works bad.

    I have run this for all three Pandora services and they all currently say "running" (Pandora Server, Pandora Agent, and Tentacle Server)

    
    [root@pandora plugin]# /etc/init.d/pandora_server status
    Pandora FMS Server is running with PID 3580.
    
    [root@pandora plugin]# /etc/init.d/pandora_agent_daemon status
    Pandora FMS Agent is running with PID 3080.
    
    [root@pandora plugin]# /etc/init.d/tentacle_serverd status
    Tentacle Server is running with PID 3145.
    

    With pluginserver set to "1" in the pandora_server.conf what does this actually do ? Is there a process I can see running that schedules the various plugins to run ? Should the processes of schedules be listed in a DB table somewhere ? Should the pluginserver output anything to any log file ? Thanks. Jon.

    1. You can try to use the command Chkconfig to add the mysql service to the list of services that starts on boot.

     

    I'll try and add mySQL to this list, but mySQL *is* starting on boot. I just guess its taking time to start up. It is Pandora Server that is not starting on boot. It complains it cant connect to mySQL. But when I start it manually a few minutes later, it starts ok. I will try and add some dependency on mySQL to the Pandora startup parameters.

    [root@pandora plugin]# chkconfig
    
    netconsole              0:off   1:off   2:off   3:off   4:off   5:off   6:off
    network                 0:off   1:off   2:on    3:on    4:on    5:on    6:off
    pandora_agent_daemon    0:off   1:off   2:on    3:on    4:on    5:on    6:off
    pandora_server          0:off   1:off   2:on    3:on    4:on    5:on    6:off
    tentacle_serverd        0:off   1:off   2:on    3:on    4:on    5:on    6:off
    
    1. Can you please Force manually the module of any of the plugins to see if it works?¿ Can you revise the configuration of the plugin? Can you see if in /etc/pandora/pandora_server.conf the "pluginserver" is on 1?

    When I force the plugin, it gets stuck at the "refresh" icon for a long time and never executes. As I write this all three I clicked "force" on are still showing the circular arrow "refresh" icon, and its been almost 20 minutes ? I can edit the config and the changes update successfully. pluginserver is set to "1" in the pandora_server.conf. I just tried adding a new script in to the plugin directory. I registered it under "Servers" - "Plugins" and it created ok. I then added the server plugin to one of the agent pages under modules, and when I click "create" the new module is stuck blue in "not initialised" ?

    1. Try to use /etc/init.d/status instead of systemctl. Sometimes i systemctl works bad.

    I have run this for all three Pandora services and they all currently say "running" (Pandora Server, Pandora Agent, and Tentacle Server)

    
    [root@pandora plugin]# /etc/init.d/pandora_server status
    Pandora FMS Server is running with PID 3580.
    
    [root@pandora plugin]# /etc/init.d/pandora_agent_daemon status
    Pandora FMS Agent is running with PID 3080.
    
    [root@pandora plugin]# /etc/init.d/tentacle_serverd status
    Tentacle Server is running with PID 3145.
    

    With pluginserver set to "1" in the pandora_server.conf what does this actually do ? Is there a process I can see running that schedules the various plugins to run ? Should the processes of schedules be listed in a DB table somewhere ? Should the pluginserver output anything to any log file ? Thanks. Jon.

    Good morning Jon, The pluginserver set to 1 enables the monitoring with the server plugins. The process that runs when monitoring with a plugin is just Pandora_server.. so you can not see the schedules. The monitoring of the plugin can be outputed in the log file of the pandora_server if you have the "verbosity" parameter at 10 (not recommended because it affects the performance). Can you please attach me a screenshot of the configuration of 1 of your plugins in the registration of Plugins in the server please. Alberto

  5. Hi Jon, I'm facing a similar problem. How did you bring the server up?

  6. Hi Jon, I'm facing a similar problem. How did you bring the server up?

    Hi. I had to power down the server in the AWS console and restart it. When it came up, mySQL took time to recover, and I had to start Pandora manually. My hostname had also reset back to the default AWS one. I reset that. After that everything was back running, except the server plugins. Hope this helps, Regards, Jon.


    Good morning Jon, The pluginserver set to 1 enables the monitoring with the server plugins. The process that runs when monitoring with a plugin is just Pandora_server.. so you can not see the schedules. The monitoring of the plugin can be outputed in the log file of the pandora_server if you have the "verbosity" parameter at 10 (not recommended because it affects the performance). Can you please attach me a screenshot of the configuration of 1 of your plugins in the registration of Plugins in the server please. Alberto

    Hi. I've turned on verbose logging. I'll leave it for a day and check the logs tomorrow. Screenshots of both configs attached. Thanks ! Regards, Jon.

    [

    ](//pandorafms.com/community/media/forum/10841/27473=2636-Plugin-Config.png)

    [

    ](//pandorafms.com/community/media/forum/10841/27473=2637-Monitor-Config.png)

  7. Good morning Jon, The pluginserver set to 1 enables the monitoring with the server plugins. The process that runs when monitoring with a plugin is just Pandora_server.. so you can not see the schedules. The monitoring of the plugin can be outputed in the log file of the pandora_server if you have the "verbosity" parameter at 10 (not recommended because it affects the performance). Can you please attach me a screenshot of the configuration of 1 of your plugins in the registration of Plugins in the server please. Alberto

    Hi. I've turned on verbose logging. I'll leave it for a day and check the logs tomorrow. Screenshots of both configs attached. Thanks ! Regards, Jon.

    Good morning Jon, The configuration of the plugins seams alright.. lets wait to see the log and watch what is happening.. Alberto

  8. Hi Alberto, So, I have reviewed the log files. I'm happy to attach them if it will help. I removed lots of repetitive entries, and noticed some things. I have this in pandora_server.log :

    
    2019-02-03 18:09:28 pandora [V10] Decoding json macros from # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh'
    2019-02-03 18:09:28 pandora [V9] Executing AM # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh pandora.xxx.co.uk'
    2019-02-03 18:09:28 pandora [V10] Processing module 'SSL Expiry - pandora.xxx.co.uk' for agent ID 12.
    
    2019-02-03 18:14:33 pandora [V10] Decoding json macros from # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh'
    2019-02-03 18:14:33 pandora [V9] Executing AM # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh pandora.xxx.co.uk'
    2019-02-03 18:14:33 pandora [V10] Processing module 'SSL Expiry - pandora.xxx.co.uk' for agent ID 12.
    

    So, whatever runs the server modules *is* running. One of my modules is running successfully every 5 minutes. However, the same module configured for other agents is not, and my other modules are not. There is nothing in the pandora_server.error log. It is full of

    Use of uninitialized value in concatenation (.) or string at /usr/lib/perl5/PandoraFMS/Core.pm line 4877.
    

    Other than that, there is a single line :

    2019-01-22 22:53:31 - pandora Starting Pandora FMS Server. Error logging activated.
    

    So, I guess the question is, why are my modules not running, if the engine is running and one other module is running. I have tried deleting them and recreating them, but they don't work either. Is there a query I can run against the DB to list all the modules configured to run ? Maybe my DB is corrupt ? Thanks. Jon.

  9. Hi Alberto, So, I have reviewed the log files. I'm happy to attach them if it will help. I removed lots of repetitive entries, and noticed some things. I have this in pandora_server.log :

    
    2019-02-03 18:09:28 pandora [V10] Decoding json macros from # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh'
    2019-02-03 18:09:28 pandora [V9] Executing AM # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh pandora.xxx.co.uk'
    2019-02-03 18:09:28 pandora [V10] Processing module 'SSL Expiry - pandora.xxx.co.uk' for agent ID 12.
    
    2019-02-03 18:14:33 pandora [V10] Decoding json macros from # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh'
    2019-02-03 18:14:33 pandora [V9] Executing AM # 234 plugin command '/usr/share/pandora_server/util/plugin/sslcheck.sh pandora.xxx.co.uk'
    2019-02-03 18:14:33 pandora [V10] Processing module 'SSL Expiry - pandora.xxx.co.uk' for agent ID 12.
    

    So, whatever runs the server modules *is* running. One of my modules is running successfully every 5 minutes. However, the same module configured for other agents is not, and my other modules are not. There is nothing in the pandora_server.error log. It is full of

    Use of uninitialized value in concatenation (.) or string at /usr/lib/perl5/PandoraFMS/Core.pm line 4877.
    

    Other than that, there is a single line :

    2019-01-22 22:53:31 - pandora Starting Pandora FMS Server. Error logging activated.
    

    So, I guess the question is, why are my modules not running, if the engine is running and one other module is running. I have tried deleting them and recreating them, but they don't work either. Is there a query I can run against the DB to list all the modules configured to run ? Maybe my DB is corrupt ? Thanks. Jon.

    Good morning JonRussell, Can you please attach me the configuration of the module that works, and one that doesn't work please. Here is a query to see what modules are in you database: select * from tagente_modulo, use in in the Interface of Pandora FMS. You can see the status of your database in Diagnostic tool. Alberto

  10. select * from tagente_modulo where descripcion LIKE '%SSL%'
    

    Results from query attached. (hostnames have been changed to protect the innocent :-) I cant see any differences between them ? module #234 is running fine, and I have live data. All the others are "Unknown" for 23 days. module #239 is me creating a new module. Thats not running either. I edited the script :

    
    [root@pandora]# cat /usr/share/pandora_server/util/plugin/sslcheck.sh 
    #!/bin/bash
    
    echo `date` $1 >> /tmp/sshcheck.log
    
    if (($# != 1));
    then
            echo "Syntax:  "
            exit -1
    else
    ...
    

    and the log just contains the one that's running (#234). So, the script is definitely not being called for the other 5 instances ?

    
    [root@pandora]# cat /tmp/sslcheck.log
    Tue Feb 5 12:26:19 UTC 2019 pandora.xxxx.co.uk
    Tue Feb 5 12:31:24 UTC 2019 pandora.xxxx.co.uk
    Tue Feb 5 12:36:29 UTC 2019 pandora.xxxx.co.uk
    

    Thanks. Jon.

    SQL Results.xlsx

  11. select * from tagente_modulo where descripcion LIKE '%SSL%'
    

    Results from query attached. (hostnames have been changed to protect the innocent :-) I cant see any differences between them ? module #234 is running fine, and I have live data. All the others are "Unknown" for 23 days. module #239 is me creating a new module. Thats not running either. I edited the script :

    
    [root@pandora]# cat /usr/share/pandora_server/util/plugin/sslcheck.sh 
    #!/bin/bash
    
    echo `date` $1 >> /tmp/sshcheck.log
    
    if (($# != 1));
    then
            echo "Syntax:  "
            exit -1
    else
    ...
    

    and the log just contains the one that's running (#234). So, the script is definitely not being called for the other 5 instances ?

    
    [root@pandora]# cat /tmp/sslcheck.log
    Tue Feb 5 12:26:19 UTC 2019 pandora.xxxx.co.uk
    Tue Feb 5 12:31:24 UTC 2019 pandora.xxxx.co.uk
    Tue Feb 5 12:36:29 UTC 2019 pandora.xxxx.co.uk
    

    Thanks. Jon.

    Good morning JonRussell, As you said, it seams to be all Ok with the same data as the one thats working. The only thing I can think about is an error in the configuration of the modules that are not the 234. If its not to much, can yoy attach me: 1- Configuration of the module working and 1 not working. 2- Configuration of the agent of the module working and 1 of the not working. It must be a little difference between one and the other that makes that failure.. Alberto

  12. Hi Alberto. Attached are the configs. However, I'm not convinced its a config error (although I do hope it is) because when I create a new module they don't work either ? But I hope you find something wrong in the configs. Thanks. Jon.

    [

    ](//pandorafms.com/community/media/forum/10841/27669=2678-SSLCheck-Working.png)

    [

    ](//pandorafms.com/community/media/forum/10841/27669=2677-SSLCheck-Not-Working.png)

    [

    ](//pandorafms.com/community/media/forum/10841/27669=2676-Agent-Working.png)

    [

    ](//pandorafms.com/community/media/forum/10841/27669=2675-Agent-Not-Working.png)

  13. Hi Alberto. Attached are the configs. However, I'm not convinced its a config error (although I do hope it is) because when I create a new module they don't work either ? But I hope you find something wrong in the configs. Thanks. Jon.

    Good morning JonRussell, In the Agent that is not working you aren't pointing to any server to do the checks, so it doesn't know how to do the monitoring. Please try to put on the pandora server on the agent and please tell us the results. Alberto

  14. HA ! That's fixed it ! Thank you so much... :-) All my agents and modules are now green ! Any idea why this might have happened suddenly after the AWS network failure ? I certainly didn't go and uncheck them all ! :-) Thanks anyway. Regards, Jon.

  15. HA ! That's fixed it ! Thank you so much... :-) All my agents and modules are now green ! Any idea why this might have happened suddenly after the AWS network failure ? I certainly didn't go and uncheck them all ! :-) Thanks anyway. Regards, Jon.

    Hello JonRussell, The only thing it occurs to me it is the server name was changed after AWS failure, and the agents didn't recognize the new one again. Or the agents were uploaded manually without the Server in their parameters. Regards

Entra en tu cuenta para publicar.