Brandon

Apache Solr datasources need scrubbing

Recommended Posts

I'm not sure where else to post this, but I wanted to make sure the LogicMonitor team takes another look at the apache solr datasources in the repository as they need to be scrubbed.  Currently these datasources use applies to rules that seem to be very specific to one customer.  Here is one example:

contains(system.groups,"SEARCH/CHI/SLAVE/JOB") or contains(system.groups,"SEARCH/AMS/SLAVE/JOB") or contains(system.groups,"SEARCH/QTW/SLAVE/JOB") or contains(system.groups,"SEARCH/CHI/MASTER/JOB") or contains(system.groups,"SEARCH/AMS/MASTER/JOB")
or contains (system.groups,"SEARCH/QTW/SLAVE/JobAux")
or  contains(system.groups,"SEARCH/CHI/SLAVE/JOBAUX")
or system.displayname =~ "solrone"

I should also mention that I have completely rebuilt these datasources and combined them into a single datasource that pulls all of this data with a single API query.  I'll be submitting it after I scrub and polish it. :)

  • Like 1

Share this post


Link to post
Share on other sites

Brandon, thanks for pointing this out.  The AppliesTo field should have been set to look for the existence of the jmx.port property.  It has been fixed.

Share this post


Link to post
Share on other sites
5 minutes ago, George Bica said:

Where do we set the jmx.port property, in each device instance?

 

Where do we set the jmx.port property, in each device instance? and what should the value be? The standard SOLR port is 8983

Share this post


Link to post
Share on other sites

This datasource doesn't poll SOLR via the API.  It uses JMX which has to be enabled on your SOLR cores (collections) in solrconfig.xml.  You can read how to do that here:  https://lucene.apache.org/solr/guide/6_6/using-jmx-with-solr.html

JMX provides lots of data that might not be exposed via the standard api queries.  However, SOLR is actually quite good about making sure all sorts of monitoring data is available without JMX and they don't recommend enabling it in production.

Share this post


Link to post
Share on other sites

We have the following SOLR datasources imported but they are not finding the instances.

  • Solr JVM Stats
    • Applies to: solr.port && solr.append
  • Apache_SOLR_* (many)
    • applies to: jmx.ports

How do we make the Apache_SOLR_* datasources recognize SOLR instances? They seems to depend on the jmx.ports value, but 8983 is not correct.

Edited by George Bica
further details

Share this post


Link to post
Share on other sites

@George Bica the jmx.ports property must be set on the devices manually. The value of the property should be your JMX port(s) separated by commas. Once you've done this, the Apache_SOLR* modules should apply and start collecting. "Solr JVM Stats" has since been deprecated.

JMX was probably selected over the API for expediency, or perhaps that API wasn't around then.

 

Share this post


Link to post
Share on other sites

@George Bica - I cooked this up and have been steadily improving it over time.  I figured I might as well share it since it seems to be causing you some pain.  :)

This datasource polls SOLR via the API and port.  On the device, you'll need to set up these two properties:

  • solr.port (default is 8983)
  • solr.append (default is solr)

The scripts I put together are pretty efficient, so discovery and monitoring should only take a minute or two to get going.  If it still doesn't work, you might be running into a firewall issue or something.

I've got more SOLR datasources I'll be posting in the next few minutes so keep an eye out.  Though - you might not be able to pull them down until they're cleared by the fine folks at LogicMonitor.

 

Share this post


Link to post
Share on other sites

@Michael Rodrigues - Thanks!  I've got more on the way.

@George Bica - Here's another one.  It might not work for your version of SOLR, but it has the exact same requirements otherwise.  Add solr.port and solr.append to your solr servers and this datasource will provide lots of useful JVM metrics without the need for enabling JMX.

I'm still going through my datasources, so I might still have one or two to post.  I'll keep replying to this thread with whatever I've got.

 

Share this post


Link to post
Share on other sites

@George Bica - Last one.  This is an eventsource, so no pretty graphs here.  It makes an API call to pull all of the events in the solr log and then alerts on Error and Severe events only.  It doesn't apply to servers by default because it can be quite noisy if you don't have it tuned properly.  Once you're sure it's not going to blow up after you've applied it, go ahead and change the Applies To rules and you should be good to go.  Hope these help!

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.