Gary Dewrell

Automated RBL Check

Recommended Posts

I know this is out there :)
I would love Logicmonitor to have the ability to check IPs for inclusion in the most popular RBL list. 
Sometimes for one reason or another we will find that one of our mail IPs ends up on some RBL lists. Takes little time to track down and get removed. 

I know their are services like https://www.rblmon.com/packages/ that we could sign up for but I am a big fan of having all my monitoring in one place. 

Is this crazy?

Share this post


Link to post
Share on other sites

Not crazy, but LM has a limited communication channel for problems.  Would be best if event sources could do it, but really not well suited due to inability to acknowledge (no correlation across checks).  If a DS was used, could program the list as instances and at least then know what triggered.  Ideally it would be a preset list, but one that is adjustable.  I am not aware of any way to do that (hybrid AD and manual instance definition), so it may require fully manual definition of the list, or AD with an external instance fetch (e.g., JSON source).

Share this post


Link to post
Share on other sites
5 hours ago, mnagel said:

Not crazy, but LM has a limited communication channel for problems.  Would be best if event sources could do it, but really not well suited due to inability to acknowledge (no correlation across checks).  If a DS was used, could program the list as instances and at least then know what triggered.  Ideally it would be a preset list, but one that is adjustable.  I am not aware of any way to do that (hybrid AD and manual instance definition), so it may require fully manual definition of the list, or AD with an external instance fetch (e.g., JSON source).

Even manual would not be terrible. Generally you are only talking about MX record IP addresses. In my case it would be about 10 items. 

 

Share this post


Link to post
Share on other sites

I am looking at the DNSLookup- DS as a starting point.  The code we used to run for Nagios was basically this Perl fragment, need to rework a bit :)

my $res = Net::DNS::Resolver->new;
my $lookupip = $host;
$lookupip =~
    s/([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})/$4.$3.$2.$1/;
for my $bl (@bls) {
    $socket{$bl} = $res->bgsend("$lookupip.$bl", 'A');
}

# watch for results to come in up to $TIMEOUT-2 seconds
my $start_time = time;
while (keys(%socket) and time - $start_time < ($TIMEOUT-2)) {
    for my $bl (keys(%socket)) {
        if ($res->bgisready($socket{$bl})) {
            my $packet = $res->bgread($socket{$bl});
            delete $socket{$bl};
            for my $rr ($packet->answer) {
                if ($rr->type eq "A" && $rr->address) {
                    $listed{$bl}++;
                }
            }
        }
    }
    if (keys(%socket)) {
        sleep(1);
    }
}

if (keys(%listed) == 0) {
    $state = 'OK'
}
elsif (scalar(keys(%listed)) < $critcount) {
    $state = 'WARNING'
}
else {
    $state = 'CRITICAL'
}

Share this post


Link to post
Share on other sites

Ok, I wrote a powershell script that will check a list of IPs against rbl-check.org and return any RBLs that have the given IP listed. 

$IPList = "x.x.x.x","y.y.y.y"  # The list will need to be supplied by Lomo. 
foreach($IP in $IPList){
    $result = Invoke-WebRequest http://rbl-check.org/rbl_api.php?ipaddress=$ip -UseBasicParsing
    $rbl = $result.content.split("`r`n") | Select-String -Pattern '\;listed'
    if($rbl){$output = $rbl}
}
if($output){return $output}else{return 0}
 
Now I have to go re-learn how to write custom datasources and covert the above to work with Lomo. Then I think I have what I waned. Automated alerts if a supplied IP is included on any RBL list. (At least any RBL list that rbl-check.org  searches.)

Share this post


Link to post
Share on other sites
1 hour ago, Gary Dewrell said:

Ok, I wrote a powershell script that will check a list of IPs against rbl-check.org and return any RBLs that have the given IP listed. 

$IPList = "x.x.x.x","y.y.y.y"  # The list will need to be supplied by Lomo. 
foreach($IP in $IPList){
    $result = Invoke-WebRequest http://rbl-check.org/rbl_api.php?ipaddress=$ip -UseBasicParsing
    $rbl = $result.content.split("`r`n") | Select-String -Pattern '\;listed'
    if($rbl){$output = $rbl}
}
if($output){return $output}else{return 0}
 
Now I have to go re-learn how to write custom datasources and covert the above to work with Lomo. Then I think I have what I waned. Automated alerts if a supplied IP is included on any RBL list. (At least any RBL list that rbl-check.org  searches.)

 

That works, just means you know "yes I am included, or no I am not included" and then you send folks to the URL in the alert.  Not the end of the world, but would be nice to know what is wrong directly.  May be possible to use this API (sparingly, I assume they would be less happy if you hit it too often so for any given IP) to discover the RBLs as instances, but then you would have to define the IP to check as a device with the DS applied.  I suppose the count of hits might have to be enough, then the instances are the target IPs, arbitrarily tied to a device, probably a collector (like Ping Multi).  I whipped one up in Groovy (stealing a bit from another DS to get the "get a URL" code -- another great use case for code libraries...).

image.png.b9dbf2496c17ddc58c2c50b0fda6f265.png

I posted my current code as 37XXA4, but it often takes a long time to get those released.

Share this post


Link to post
Share on other sites

Yea ideally you would want to return which IP failed which RBL services. As you said, maybe an event source. 
Be awesome if you could somehow take an text output from the script and have it embedded in a customer alert message then you could provide the additional information. 

Anyway, I do not know groovy at all. Any chance you could share with me your source so I can learn and play. Totally understand if not. 

Thanks!

Share this post


Link to post
Share on other sites
9 minutes ago, Gary Dewrell said:

Yea ideally you would want to return which IP failed which RBL services. As you said, maybe an event source. 
Be awesome if you could somehow take an text output from the script and have it embedded in a customer alert message then you could provide the additional information. 

Anyway, I do not know groovy at all. Any chance you could share with me your source so I can learn and play. Totally understand if not. 

Thanks!

I cannot attach it here, but please copy from the link below (will need to setup keybase, but if you haven't, you should :)).

keybase://public/ciscoqid/LogicMonitor/RBLCheckMulti-.xml

It is still entirely done, but it is working. I need to add some documentation and tune it a bit.  For this, I set applies to on all collectors, then you use the "Add Monitored Instance" on whichever collectors you want.  The instances should have the hostname and IP as name and wildvalue, respectively.  I did not code support (yet) for using a FQDN as the wildvalue, just IP.

Share this post


Link to post
Share on other sites
1 minute ago, mnagel said:

I cannot attach it here, but please copy from the link below (will need to setup keybase, but if you haven't, you should :)).

keybase://public/ciscoqid/LogicMonitor/RBLCheckMulti-.xml

It is still entirely done, but it is working. I need to add some documentation and tune it a bit.  For this, I set applies to on all collectors, then you use the "Add Monitored Instance" on whichever collectors you want.  The instances should have the hostname and IP as name and wildvalue, respectively.  I did not code support (yet) for using a FQDN as the wildvalue, just IP.

Thank you very much. Not sure the service I was using supported hostname. It specifically says IP address on the site. 

 

Share this post


Link to post
Share on other sites

Right, you must use IP. I just thought it might be nice to specify a hostname and have the DS resolve that for submission to the API.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.