Purnadi K

LogicMonitor Staff
  • Content count

  • Joined

  • Last visited

Community Reputation

0 Neutral

About Purnadi K

  • Rank
    Community Whiz Kid
  1. Info: Groovy script is the base for the environment and certainly the target/end device must be a linux-based, be it a distro of Linux server/desktop or an appliance with linux-kernel firmware (err...it's basically almost every device then: firewalls, switches, IDS/IPS, LB, Apple devices, Android devices, etc....it seems all gadgets on the planet using linux kernel) It is a request some years ago (seems so long though, which actually is +2 years) by a LogicMonitor's Customer: Obviously, this is not an official datasource crafted by the amazing Monitoring Engineering team of LogicMonitor, but patting my own back, it suffices to say that it serves the purpose for a better security connecting remotely which has been a common best-practice by anyone who enjoys text-based command line remote session. SSH Key is used over the option of sending password due to the apparent reason of information security, although one might argue that the ssh password will be only between a LogicMonitor collector and the endpoint, within a supposed-to-be internal network. Yet a security best practice may dictate such usage regardless of the network. Before progressing further, however, there is a catch in using ssh key, which is the necessity for a deployment of public key to each target device. Simply put, every time SSH Keys are used for remote session between two devices, there will be private key and public key used for authentication process, hence no password needed. These keys are to be put in the devices, private key in the source device where the remote session is originated and public key in the endpoint. Private key is not for a share whilst public key is for any device that the source will need to connect, should the ssh key be used. The only hassle, even if it is considered to be one, is to load that public key on each target device (if there are many). From the security standpoint, that is totally not an issue and rather a compulsory instead. (As a comparison, by using ssh user and password, the process would be similar too, that is to create user and password in each target device). This practice is really not an ancient stuff and almost every cloud provider, AWS being the one, has that feature recommended (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html). In LogicMonitor case where I did the test, it is between a collector server/device and a monitored device (both happen to be Ubuntu 16 although Windows can be used for the collector as well and on the other hand, definitely it can not be used for Windows target device, for the obvious reason). For my simple test, it is just to monitor a log file size which is a syslog. One thing worth noting is, remote session type of monitoring will certainly consume more resource of collector, as a matter of fact, every datasource using script will do. Besides, using this method, the processing time seems to increase by a little bit, compared with user/password, but I have not done any thorough observations though (not only that I do not intend to, since this is just a test, nor have I the environment huge enough to do a high load-test). Security and processing speed, they do not go in parallel for sure, especially considering the recent havoc by a processor company caused a nightmare for information security worldwide, bypassing security measure for the sake of increasing a speed of data processing. So here is the script which is basically running a command to throw output of a data from a file named 'dusyslog' in the remote device and a datapoint will capture it (datapoint name: size): import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.util.Properties; import com.jcraft.jsch.Channel; import com.jcraft.jsch.ChannelExec; import com.jcraft.jsch.JSch; import com.jcraft.jsch.Session; try{ String command = "cat dusyslog"; String host = hostProps.get("system.hostname"); String user = hostProps.get("ssh.user"); String key = hostProps.get("ssh.key"); JSch jsch = new JSch(); jsch.addIdentity(key); jsch.setConfig("StrictHostKeyChecking", "no"); Session session=jsch.getSession(user, host, 22); session.connect(); Channel channel = session.openChannel("exec"); channel.setCommand(command); channel.setErrStream(System.err); channel.connect(); InputStream input = channel.getInputStream(); channel.connect(); try{ InputStreamReader inputReader = new InputStreamReader(input); BufferedReader bufferedReader = new BufferedReader(inputReader); String line = null; while((line = bufferedReader.readLine()) != null){ println(line); } bufferedReader.close(); inputReader.close(); }catch(IOException err){ err.printStackTrace(); } channel.disconnect(); session.disconnect(); }catch(Exception err){ err.printStackTrace(); } The first thing you can notice is I am using : jsch.addIdentity(key) for adding the key file into the session for authentication. So what is this file? it is the private key file residing in a secure place in collector server. You need to make sure the file 'dwelleth in the secret place of the most High and shall abide under the shadow of the Almighty'.I really mean it that the private key should not be exposed to the world. But of course, I make it sound like a very serious security matter So just to make sure that file is given limited permission for collector service to access is sufficient. Undoubtedly the script is not built by myself from scratch but I have made some modification so it is safe to be copyrighted by me and you have the 'the right to copy' & enhance it if you need to and providing me a treat of coffee would be highly appreciated. Further to that, this part: key = hostProps.get("ssh.key"); as per normal, is defined at the device property and following is the sample from my test: Linux device: /security/key Windows device: C:\\security\\key Note: you can add an additional security to disguise the location of the file too and that folder "security" is not the original folder where the private key resides. This is for paranoid security practitioners. (But as I usually joke with friends in the IT field, the best data security was during WordStar era, before Windows 3.11 came and before tcp/ip was introduced to home users ). Below are some screenshots from the implementation:
  2. Ssh key based auth for Groovy Scripts Datasources

    hi Martin, I happened to come across your inquiry. Just want to let you know that you can use ssh key to log into your device from collector. I have tried it and it worked well. But you need to of course copy the public key into each target device. Usual drill as you might have known.
  3. Being a newbie myself in the area of PowerShell, this is just a simple example or even 'raw' codes of PS for LogicMonitor REST API which hopefully it will be of little benefit to whomever needs it. I believe one of many advantages using LogicMonitor is the freedom that the other products in the market may not give and that is to deploy creativities in the existing monitoring. The example provided here is pertaining to REST API where LogicMonitor has a bunch of publication on our help page: and several code samples are inclusive in the documents. I have seen many of our great Clients have used them and even made it to next level where even our Support team (or probably limited to only myself) was 'brought to our knees' , and must admit the very advanced level these precious Clients have. What an excellent people they are! Recently we had one nice client who asked for code sample to retrieve device in the portal, and not only few but ALL of them. We do have a sample code, kind courtesy of our Product Team (Ms. Sarah Terry): If the request is to use Python, that sample would have been sufficient, nonetheless, or I should say 'unfortunately', it requires PowerShell. As we all know that our Support Team's standard response would be to suggest our Clients making in-house development if such requirements arise or wait for our Product/Development to release a fresh sample in the abovementioned help page. But on this occasion, using the 'cheat' sheet in the other samples, I tested the following code which gives me a satisfactory result (for a newbie certainly): <# account info #> $accessId = '.....[API access id].....' $accessKey = '....[API access key].....' $company = '.....[LogicMonitor portal/account]....' $allDevices = @() <# loop #> $count = 0 $done = 0 While ($done -eq 0){ <# request details #> $httpVerb = 'GET' $resourcePath = '/device/devices' $queryParams ='?offset=' + $count + '&size=3'+'&fields=id,displayName' <# Construct URL #> $url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams <# Get current time in milliseconds #> $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds) <# Concatenate Request Details #> $requestVars = $httpVerb + $epoch + $resourcePath <# Construct Signature #> $hmac = New-Object System.Security.Cryptography.HMACSHA256 $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey) $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars)) $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-' $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower())) <# Construct Headers #> $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" $headers.Add("Authorization",$auth) $headers.Add("Content-Type",'application/json') <# Make Request #> $response = Invoke-RestMethod -Uri $url -Method Get -Header $headers $total = $response.data.total $devId = $response.data.items.id $numDevices = $devId.Length $count += $numDevices $items = $response.data.items $allDevices += $items if ($count -eq $total) { $done = 1 } } #end loop Write-Host ($allDevices| ConvertTo-Json -Depth 5) So, hopefully, somebody can sanitize the codes for the better for their own usage. The important highlights are: that will need to be changed accordingly, and: that depends on what fields to be displayed in the output, how many number of devices to be retrieved at one time (size=3). Note: using dummy devices in our lab portal, 11 devices can be successfully retrieved by iterating 3 at a time for 4xAPI requests. I never tested with above 1000 devices as the document mentioned to be the limit per request (although it would be nice if any of our Clients is willing to test. If only Google be our Client, great to test their two freshly-built datacenters at the far-west of Singapore with 'must-be' more than thousands of devices) :
  4. Note: As of the publication of this article, collector 25.000 is a GD (optional general release), which means this article will be obsolete as version goes forward. In the past 2-3 months I had two cases whereby error occurred when an Internal Service Check of a website is authenticated with NTLM using ADFS. That error seemed odd with a message of: or in the detailed response, it can be seen as: <title>401 - Unauthorized: Access is denied due to invalid credentials.</title> regardless whether the credentials (username,password) set in the Service Check configuration are correct. Based on the design by Product & Development team, previous collector version (before 24.300), the error is "normal" due to the fact that the URL of the request origin is different from authentication URL, which in this case is ADFS URL and the collector does not pass the credentials to the authentication server which makes the process fails. Fortunately with the arrival of version 25.000, this all has been changed so redirected authentication will be supported as explained in this document: (see "General Deployment Collector - 25.0") It is evident with my little test that you may also see in the screenvideo below: website to check: http://admin.lmglobalsupport.com (redirected to http://pk.lmsupportteam.com) ADFS authentication: https://fspk.lmsupporteam.info The following is additional screenshots of the location in IIS (which I used for my test) to configure the HTTP redirection: Here is just a preview about website authentication in a browser:
  5. As we all know, LogicMonitor has a monitoring to check if web server is alive or responding in a device. It uses datasource called HTTP(S) with a nice short note here: https://www.logicmonitor.com/support/datasources/data-collection-methods/webpage-httphttps-data-collection/ However, there is a catch :). and explanation is on the way.... I am running an NGINX (with SSL). This datasource is supposed to correctly monitor the secured web server but it is not and there is an error code 5 and based on the note in datasource it means network error. What kind of error is that? It will demand a closer look. ( Firstly, such error usually will lead to some checks if the web server is running fine and with LogicMonitor Collector debug, there is a tool to do just that: !http In the debug window, it shows that the web server has no issue. being paranoid .. or in fact, a standard procedure for any devops/sysadmin(?), checking the web server service rightfully to be done as well and it is totally up and running. Further to that, querying the web server from the collector server shell is another evidence that it is all well with the web server. Checking from collector debug? checking the web server? query from collector shell to the web server? This was very confusing at first until a more detailed investigation was done. HTTP protocol version is the centre of this blunder. The datasource is sending a request with HTTP version 1.0, while the server does not accept it and it throws error every time. The web server can be configured to accept both the older protocol 1.0 (which is still widely used) and 1.1, nevertheless, some of them only accept 1.1 Collector debug command !http in LogicMonitor, does not specifically request with 1.0, therefore there is no error when such debug is run. The result shows as evidence: It also leads to the finding that the web server return the query in protocol version 1.1 If the query specifies which protocol to use such as in datasource HTTPS, the web server will cough up that error: Server2000:~$ curl -k -I --http1.0 https://test-server_nginx curl: (52) Empty reply from server and when the request is specified with 1.1, the result is just what it should be: Server2000:~$ curl -k -I --http1.1 https://test_server_nginx HTTP/1.1 200 OK Server: nginx/1.10.0 (Ubuntu) Date: Sun, 13 Aug 2017 09:59:35 GMT Content-Type: text/html Content-Length: 6031 Last-Modified: Wed, 10 May 2017 11:51:16 GMT Connection: keep-alive ETag: "5912feb4-178f" Strict-Transport-Security: max-age=63072000; includeSubdomains X-Frame-Options: DENY X-Content-Type-Options: nosniff Accept-Ranges: bytes Therefore the datasource needs to be modified to deal with web servers that do not like the query with the earlier version of 1.0, with the following on the request part: GET / HTTP/1.1 \nUser-Agent:LogicMonitor\nConnection: close\n\n hence the monitoring will be corrected: As a note, there is a configuration in NGINX whereby it will strict the web server to only respond to 1.1 protocol and reject 1.0 which is the following: if ($server_protocol ~* "HTTP/1.0") { return 444; } "444" is the code that will be provided when the query is using 1.0, which means: 444 No ResponseUsed to indicate that the server has returned no information to the client and closed the connection.495 SSL Certificate Error The error in LogicMonitor will correspond to that error configuration in the web server and in this case when protocol 1.0 is used, it returns code 5. For IIS, the restriction will be in web.config as follows, which is using URL Rewrite module in IIS: <rules> <rule name="rulename1" patternSyntax="Wildcard" stopProcessing="true"> <match url="*" /> <conditions> <add input="{SERVER_PROTOCOL}" pattern="HTTP/1.0" /> </conditions> <action type="AbortRequest" /> </rule> </rules> </rewrite> </system.webServer>
  6. Simple Check for SSL Cert Expiration Monitoring

    hi Kaushal, you can use this datasource if you can add the ELB as a device.
  7. Monitoring Tomcat Context with HTTP Data Collection Tomcat is not a common home pet that people normally know. It is not a type of a cat ("a male cat") after all but an eccentric name like any other open source software's names i.e. Guacamole, Apache, RabbitMQ, etc. which I believe represents the freedom nature of those software, where creativity is the main ingredient. I am not here to explain about Tomcat per se, either because of Google provides abundant information about it or it is out of the scope of my article. Simply put, Tomcat is a servlet whereby multiple Context containers can exist. Each context refers to a web application. Recently we have quite a few customers request to monitor the status and response time of the Tomcat Context, which is actually simple since there are already readily available HTTP Datasource or Internal Web Service Check for that purpose. However, our customers have, in this case, multiple contexts as in normal Tomcat application. Therefore Active Discovery (AD) is needed to get the list of Contexts running on Tomcat before the simple HTTP Data collection, that has only two Datapoints (Response Time & Status) will be applied. For Active Discovery part, I would need to give credit to our renowned David Lee whom you might be familiar if you ever open a ticket with LogicMonitor Support channel. For the sake of confidentiality, all the screenshots will be in my own lab instead of our client, it is, however, producing the similar intended result although there are more Context containers in the real production environment. The AD is a short groovy script utilizing Java Management Extension to connect to remote Tomcat from Collector. Discovery Method chosen should be 'SCRIPT' in this Datasource. import com.santaba.agent.groovyapi.http.*; import com.santaba.agent.groovyapi.jmx.*; import javax.management.remote.JMXServiceURL import javax.management.remote.JMXConnectorFactory import org.jboss.remotingjmx.* def jmx_host = hostProps.get('system.hostname'); def jmx_port = hostProps.get('tomcat.jmxports'); def jmx_url = "service:jmx:rmi:///jndi/rmi://" + jmx_host + ":" + jmx_port + "/jmxrmi"; context_array = jmx_conn.getChildren("Catalina:type=Manager,host=localhost,context=*"); context_array.each { context -> println context+"##"+context } return 0; (note: tomcat.jmxports is the port used by JMX to connect to Tomcat servlet and in this case is a standard port 9000) Following is the result of AD: (note: one of the context name is: 'context-test') which can be tested from the collector debug window as follow: $ !jmx port=9000 h= "Catalina:type=Manager,host=localhost,context=*" Catalina:type=Manager,host=localhost,context=* => /examples /manager /docs /context-test /host-manager / As for the data collection, the mechanism that is used to collect data is 'WEBPAGE' (Note: Port number will depend on the setting in Tomcat and the wildvalue will be the Context name) Data collection can be tested in the collector with the command !http: HTTP response received at at: 2017-04-20 09:05:30.85. Time elapsed: 3ms HTTP/1.1 200 Accept-Ranges: bytes ETag: W/"214-1492696731000" Last-Modified: Thu, 20 Apr 2017 13:58:51 GMT Content-Type: text/html Content-Length: 214 Date: Thu, 20 Apr 2017 14:05:30 GMT <html> <body> <h3>TEST Tomcat Context web access</h3> <pre> <> [ { status: "OK", context: "context-test" }, { company:"LogicMonitor", } ] </> </pre> </body> </html> Here is the final result of the monitoring: From the browser, the Tomcat context can be accessed just like a normal website: The Datasource is only available for download from LM Exchange (version 1.1). This is not available in core repository of LogicMonitor Datasource. Note: This is what Tomcat Manager looks like:
  8. Monitoring SSL Certificate expiry days can be done in LogicMonitor by making use of datasource SSLCerts- (SSL Certificate Expiration). On the side note, SSL Certificate is used for certifying a web server that does the secured socket layer data encryption between a web server and a client (web browser). SSL Certificate is issued by several organizations/companies so called Certificate Authority (CA) for the purpose of providing the legitimacy of the web servers that encrypt the data for communication. The certificates issued will be digitally-signed by those CA and can be trusted by the client based on Root Certificates installed in the common browsers. It is, however, possible to create a self-signed certificate, which in this case is used for a testing purpose. Data will still be encrypted but the certificate will not be trusted by the client browsers. When a device with SSL Cert installed has been added to LogicMonitor, rightfully that datasource will be auto-applied, as with other normal datasources, and after some collection cycles, the data of the certificate remaining days to expire should appear. Under the circumstances whereby the monitoring does not work as per normal, common recommendation will be to go through the following simple procedures: 1) Device check, whether or not the SSL Certification has been configured properly 2) Accessibility from collector 3) Data collection test from collector 1) For a start is to check if the SSL certificate configuration is properly done in the web server - Each web server may have a different way of setting up the certificate, the following is an example for NGINX & IIS: ssl_certificate "/etc/cert/nginx/private/[cert name].crt"; ssl_certificate_key "/etc/cert/nginx/private/[cert name].key"; - An open port check would be good as well with below output from the check (note: port is bound to any interfaces or possibly only one interface on the web server): Linux: tcp 0 0* LISTEN Windows: TCP LISTENING 2) The next check will be to access the web server from the collector (obviously the collector must be able to reach to the device where the web server is installed): Note: Collector debug window is needed for this check, please refer to this article: https://www.logicmonitor.com/support/settings/collectors/using-the-collector-debug-facility/ - the main command is simply: !http (help !http will give info for the command itself) $ !http HTTP response received at at: 2017-03-26 16:28:55.581. Time elapsed: 20ms HTTP/1.1 200 OK Server: nginx/1.10.2 Date: Sun, 26 Mar 2017 08:28:55 GMT Content-Type: text/html Content-Length: 5948 Last-Modified: Wed, 04 Jan 2017 08:44:56 GMT Connection: keep-alive ETag: "586cb608-173c" Accept-Ranges: bytes It shows that the web server is accessible at port 443 (HTTPS) with response code 200 as follows: 3) The last one will be to check if data can be collected from the collector which is the remaining days to the expiry of the certificate. Collector debug window is still needed for this check. For Linux collector: $ !java -cp ../lib/certexpire.jar CertificateExpire /usr/local/logicmonitor/agent 443 true Enable debug SSL cert Get the support protocol, protocols=SSLv2Hello,SSLv3,TLSv1,TLSv1.1,TLSv1.2, Get the enabled protocol, protocols=TLSv1,TLSv1.1,TLSv1.2, Try to send request to server. Request send ... TrustManager: checkServerTrusted got 1 certs. Auth type: ECDHE_RSA Exception caught - java.security.cert.CertificateException: Certificate received. Certification 1 [Type: X.509] Issue Date: Mon Jan 02 17:51:51 SGT 2017, Expiration Date: Sat Jul 01 17:51:51 SGT 2017 Got issue date - Mon Jan 02 17:51:51 SGT 2017, expiration date - Sat Jul 01 17:51:51 SGT 2017 97 For Windows collector: $ !java -cp ../lib/certexpire.jar CertificateExpire "C:\Program Files (x86)\LogicMonitor\Agent" fspk.lmsupport.com fspk.lmsupport.com 443 true Enable debug SSL cert Get the support protocol, protocols=SSLv2Hello,SSLv3,TLSv1,TLSv1.1,TLSv1.2, Get the enabled protocol, protocols=TLSv1,TLSv1.1,TLSv1.2, Try to send request to server. Request send ... TrustManager: checkServerTrusted got 1 certs. Auth type: DHE_RSA Exception caught - java.security.cert.CertificateException: Certificate received. Certification 1 [Type: X.509] Issue Date: Thu Feb 02 03:16:57 PST 2017, Expiration Date: Sat Feb 02 03:16:57 PST 2019 Got issue date - Thu Feb 02 03:16:57 PST 2017, expiration date - Sat Feb 02 03:16:57 PST 2019 660 - The basic command is: !java and complete format would be: !java -cp ../lib/certexpire.jar CertificateExpire [collector installation folder] [device name/IP address] [device name/IP address] 443 true Note: * certexpire.jar is in the library of the collector agent * device name/IP address is the web server that is registered/added into the LogicMonitor portal * collector folder is: either "C:\Program Files (x86)\LogicMonitor\Agent" or /usr/local/logicmonitor/agent The data collected can be verified on the device where the SSL Certificate is installed by accessing the web server in the browser and view the detail of the certificate loaded in the browser as follows: Having gone through all the above-mentioned checks and the results are good, it will produce this monitoring in LogicMonitor as follows: