• 0
Paul Carroll

Add device to Collector with least devices in rest api

Question

I hope someone here can help,

Using powershell and the rest API I would like to do a lookup of selected collectors find which has the least amount of devices on it then add the a new device .

I have the script for adding the device and that works great just need the collector lookup, has anyone done something similar.

Thank you in advanced for all help provided

 

Paul

Share this post


Link to post
Share on other sites

Recommended Posts

  • 0

Hi Paul,

We are actively working on an enhancement to LogicMonitor to support Load Balanced Collectors.  One of the key features is that when adding a device you can pick a Collector Group instead of an individual collector.  More to come in the next month or two!

~Forrest

Share this post


Link to post
Share on other sites
  • 1

Here is what I use to "report" on collectors and their versions.

 

$status = $null
$body = $null
$response = $null

<# Use TLS 1.2 #>
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

<# account info #>
$accessId = ''
$accessKey = ''
$company = 'SITE'

<# request details #>
$httpVerb = 'GET'
$resourcePath = '/setting/collectors'
$queryParams = '?size=1000&fields=id,description,hostname,platform,collectorGroupName,collectorSize,numberOfHosts,build'
#$queryParams = ''
$status = $null
$body = $null
$response = $null
$data = ''

<# Construct URL #>
$url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams

<# Get current time in milliseconds #>
$epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds)

<# Concatenate Request Details #>
$requestVars = $httpVerb + $epoch + $data + $resourcePath

<# Construct Signature #>
$hmac = New-Object System.Security.Cryptography.HMACSHA256
$hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey)
$signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars))
$signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-'
$signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower()))

<# Construct Headers #>
$auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$auth)
$headers.Add("Content-Type",'application/json')

<# Make Request #>
$response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers 

<# Print status and body of response #>
$status = $response.status
#$body = $response.data| ConvertTo-Json -Depth 5

$body = $response.data | ConvertTo-Json -Depth 2
$Collectors = $body | ConvertFrom-Json

#$Collectors.items | Sort collectorGroupName | format-table -AutoSize
$Collectors.items | Sort collectorGroupName | Out-Gridview
#$body

 

The numberOfHosts would help you out in this case.

Share this post


Link to post
Share on other sites
  • 0
On 13/11/2018 at 5:31 AM, Joe Williams said:

Here is what I use to "report" on collectors and their versions.

 


$status = $null
$body = $null
$response = $null

<# Use TLS 1.2 #>
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

<# account info #>
$accessId = ''
$accessKey = ''
$company = 'SITE'

<# request details #>
$httpVerb = 'GET'
$resourcePath = '/setting/collectors'
$queryParams = '?size=1000&fields=id,description,hostname,platform,collectorGroupName,collectorSize,numberOfHosts,build'
#$queryParams = ''
$status = $null
$body = $null
$response = $null
$data = ''

<# Construct URL #>
$url = 'https://' + $company + '.logicmonitor.com/santaba/rest' + $resourcePath + $queryParams

<# Get current time in milliseconds #>
$epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds)

<# Concatenate Request Details #>
$requestVars = $httpVerb + $epoch + $data + $resourcePath

<# Construct Signature #>
$hmac = New-Object System.Security.Cryptography.HMACSHA256
$hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey)
$signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars))
$signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-'
$signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower()))

<# Construct Headers #>
$auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization",$auth)
$headers.Add("Content-Type",'application/json')

<# Make Request #>
$response = Invoke-RestMethod -Uri $url -Method $httpVerb -Header $headers 

<# Print status and body of response #>
$status = $response.status
#$body = $response.data| ConvertTo-Json -Depth 5

$body = $response.data | ConvertTo-Json -Depth 2
$Collectors = $body | ConvertFrom-Json

#$Collectors.items | Sort collectorGroupName | format-table -AutoSize
$Collectors.items | Sort collectorGroupName | Out-Gridview
#$body

 

The numberOfHosts would help you out in this case.

 Thanks Joe.

Really appreciated

I have added the below which filters by selected collectors and selects the one with the least number of machines

Where-Object {($_.id -eq "CollectorId") -or ($_.id -eq "CollectorId")} | Sort-Object numberOfHosts -Descending:$false | Select-Object -First 1| Out-Gridview

Share this post


Link to post
Share on other sites
  • 0

Have you found an efficient way to get a list of devices reporting to a specific Collector?  I'm having autobalancer issues and am writing my own as a datasource so I can fire it off once a day.  I've got all of the logic solved and am just at the point where I have to get the list of devices reporting to the Collector with the most of them, then randomly select x number of them and assign them to the other collector(s) in the collector group.

Share this post


Link to post
Share on other sites
  • 0
On ‎6‎/‎21‎/‎2019 at 9:38 PM, Forrest Evans - LM said:

Hey Cole,

Are you having trouble with the newly released Auto Balanced Collectors feature?  If so please open a ticket so we can look into it and help.

~Forrest

I did... but I need the solution before you'll be able to fix it.  I ended up on a call with Casey and we walked through troubleshooting it.  The metrics being used to balance aren't evaluating correctly for our environment, so the load is unbalanced.  I am using a far simpler method of determining load to perform a balance which will reduce our VM spend (as opposed to adding more collectors to handle the load) to be able to monitor our environments.

I'm at the point in the script where I just need to grab the list of devices reporting to the Collector with the most devices associated with it, and then I'll make a random selection to cover the sum difference to redistribute and change them to the other Collector/s in the group.  We currently don't have more that 2 collectors / group, but I'm writing my script as if we did to make it more portable and save time in the future having to re-read what I'd done in the first place to refashion it for a broader use case.

Ticket # 155345 or your reference.  If you're interested to see how I'm doing what I'm doing, I'm making a DataSource (specifically so I can schedule its run) that I will applyTo our primary Collector so it has somewhere to run.  It then uses the REST API to gather data and make changes to the environment.  I'd love you guys to add a "ScriptSource" that would allow us to trigger a script based on an alert of a specific type, or on a schedule.  This would allow me to write auto-remediation scripts for common simple alerts so we could save the staffing costs to manually address them.  Right now, we have SCOM performing quite a bit of small maintenance tasks, but we'll be turning it off (replaced by LM) and it's going to get really busy with menial tasks for our technicians when we do.

Share this post


Link to post
Share on other sites
  • 0

Calling the REST API with the below should get you a list of all the devices assigned to that collector:

/device/devices?size=1000&fields=id,name&filter=preferredCollectorId:COLLECTOR_ID_HERE

Replace COLLECTOR_ID_HERE with the collector ID number which you can get from the Collector Settings page and/or via the API. If there are more than 1000 devices on a collector then you need to call this multiple times with an index as the API maxes at 1000 devices per call. I believe there are examples in the API docs.

  • Upvote 1

Share this post


Link to post
Share on other sites
  • 0

I'm still not getting any response from the API using the URL above.  Here's the response I get... the function to retrieve data works for other queries:

Invoke-RestMethod : {"errorMessage":"Authentication failed","errorCode":1401,"errorDetail":null}
At line:51 char:23
+     $response       = Invoke-RestMethod `
+                       ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], 
    WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodComma 
   nd

I get a good response with device/devices raw.  I get the above error with just the ?size=1000 as well as all of the following pieces in that query you'd posted.  I've gone so far as to separate all of the individual pieces and test them alone and no dice.  Am I using the wrong question mark?

Share this post


Link to post
Share on other sites
  • 0

I was being more general in my example. When using the LM API, you need to separate the endpoint part (/device/devices) from the query part (part after the "?") because of how LM does API authentication. Which is likely why you are getting auth failed error.

So using the example Paul provided above, you would use something like this:

$resourcePath = "/device/devices"
$queryParams = "?size=1000&fields=id,name&filter=preferredCollectorId:COLLECTOR_ID_HERE"

(You would also need to change the ending part of his script as devices have different properties than collectors.

  • Upvote 1

Share this post


Link to post
Share on other sites
  • 0

I'm at the point in my script where I'm ready to deliver the payload... and I'm informed that we don't have programmatic access to change the collectors for a device.  So that was apparently 20 hours of wasted development time - <grumble>.

@Forrest Evans - LM Is there any progress on fixing the autobalancers?  I don't see updates in my ticket to that end.

 

Share this post


Link to post
Share on other sites
  • 0

Few clarifications around the Auto Balanced Collector solution and when it chooses to balance devices:

The AutoBalance is intentionally conservative because when devices are moved between collectors there is a risk of them missing polls.  The balancing activity will not trigger a rebalance (even if the button is pushed) if the collector is not over it's own unique threshold limit.  In the current iteration, all instance types are treated equally. 

Since collectors of different sizes can handle different loads we set the default threshold to 10,000 as the number of instances threshold for 2GB of JVM memory, which corresponds to a medium sized collector.  The math to determine how many actual instances a collector can handles is the following:

(Target_Collector_Mem/Medium_Mem)^1/2 * Medium_Threshold

For example, if user sets (2G) collector's threshold to 10,000, for a large(4G) collector, the threshold will be scaled to: (4/2)^1/2 * 10000 = 14140

I checked out your current collectors and none of them are over their corresponding thresholds.  The biggest one only had 12,209 instances on a large collector.

As for adding devices if you want to add a new device to an AutoBalance Collector Group without choosing the collector the payload below would be a very minimal example to do that.  The key parts being the autoBalancedCollectorGroupID is provided as an integer and 'preferredCollectorId' is set to 0 for the device to be AutoBalanced.  This should add the device to the collector with the lowest instance count.

To create a new device POST to .../device/devices on V2 of the REST API
{
  "name": "10.1.1.22",
  "displayName": "My Device",
  "autoBalancedCollectorGroupId": 4,
  "preferredCollectorId": "0"
}
 
~Forrest

Share this post


Link to post
Share on other sites
  • 0
2 minutes ago, Forrest Evans - LM said:

I checked out your current collectors and none of them are over their corresponding thresholds.  The biggest one only had 12,209 instances on a large collector.

 

How can you determine the number of instances a collector has? Assuming this is not the same as the number of devices a collector has.

Share this post


Link to post
Share on other sites
  • 0

When you enable Auto Balanced Collectors on a Collector Group a new column is added with the instance count.

1443632713_ScreenShot2019-06-27at10_16_06AM.thumb.png.e756e5ed41578c0cb4551edceae64f02.png

I do have an enhancement on the list to always show this count rather than only on Auto Balanced Groups.

Share this post


Link to post
Share on other sites
  • 0
7 minutes ago, Forrest Evans - LM said:

When you enable Auto Balanced Collectors on a Collector Group a new column is added with the instance count.

I do have an enhancement on the list to always show this count rather than only on Auto Balanced Groups.

 

Ah thanks! I haven't played with Auto Balancing yet but it would be great to get those numbers for non-balanced collectors also.

Share this post


Link to post
Share on other sites
  • 0
23 minutes ago, Forrest Evans - LM said:

Few clarifications around the Auto Balanced Collector solution and when it chooses to balance devices:

The AutoBalance is intentionally conservative because when devices are moved between collectors there is a risk of them missing polls.  The balancing activity will not trigger a rebalance (even if the button is pushed) if the collector is not over it's own unique threshold limit.  In the current iteration, all instance types are treated equally. 

Since collectors of different sizes can handle different loads we set the default threshold to 10,000 as the number of instances threshold for 2GB of JVM memory, which corresponds to a medium sized collector.  The math to determine how many actual instances a collector can handles is the following:

(Target_Collector_Mem/Medium_Mem)^1/2 * Medium_Threshold

For example, if user sets (2G) collector's threshold to 10,000, for a large(4G) collector, the threshold will be scaled to: (4/2)^1/2 * 10000 = 14140

I checked out your current collectors and none of them are over their corresponding thresholds.  The biggest one only had 12,209 instances on a large collector.

As for adding devices if you want to add a new device to an AutoBalance Collector Group without choosing the collector the payload below would be a very minimal example to do that.  The key parts being the autoBalancedCollectorGroupID is provided as an integer and 'preferredCollectorId' is set to 0 for the device to be AutoBalanced.  This should add the device to the collector with the lowest instance count.

To create a new device POST to .../device/devices on V2 of the REST API
{
  "name": "10.1.1.22",
  "displayName": "My Device",
  "autoBalancedCollectorGroupId": 4,
  "preferredCollectorId": "0"
}
 
~Forrest

The issue we're running into isn't with the memory... it's with TCP/IP Socket exhaustion.  The number of Devices * instances was hitting the limit and we were getting 'host unreachable' messages constantly.  Moving them to an autobalanced group was supposed to alleviate it by balancing them... which it never did as it's too conservative.  I would venture a guess that your software keeps track of outstanding requests per instance/device.  If a balanced needs to occur, I would set a flag to have it movePending, as soon as the threads had all returned, then the collector move happens.

I'm looking for real balance, not just above the individual collector's threshold to get it back under the threshold.  When 2000 users log into an environment when their shift starts and all of the traffic hits an unbalanced load, one of the collectors stops delivering data, the other is fine.  That's a problem with the balancer that should be easy to fix.  I know that this kind of load is possible as we had the exact same environment being monitored by SCOM (technically, it still is - and I'm having to go back to it whenever LM fails to get telemetry).

I have guidance about how to get the collector moves to work and as soon as I'm done with the dataSource I'm building to run twice a day, I'll post that code here.

Share this post


Link to post
Share on other sites
  • 0
1 hour ago, Mike Moniz said:

I was able to update the collector on a device by using PATCH: 


$resourcePath = "/device/devices/$deviceid"
$queryParams = "?patchFields=preferredCollectorId"
$data = "{`"preferredCollectorId`":$newCollectorId}"

https://www.logicmonitor.com/support/rest-api-developers-guide/v1/devices/update-a-device/

As a test, I'm doing this:

$resourcePath = "/device/devices/2332"
$queryParam   = "?patchFields=preferredCollectorId"
$data         = "{`"preferredCollectorId`" : 6}"

It's returning with no errors, but the value doesn't seem to be changing.

Share this post


Link to post
Share on other sites
  • 0

 If you are changing the collectorId within Auto Balanced Collectors there are some other "rules" to follow.  If you want to switch a device from being Auto Balanced to directly assigned you have to set `autoBalancedCollectorGroupId` to 0 and `preferredCollectorId` to the collector ID you want it directly assigned to.   To set it back to AutoBalanced from directly assigned do the reverse: `autoBalancedCollectorGroupId` sets to id of the group and `preferredCollectorId` to 0

With Version 2 of the API you don't need the $queryparam part.  You can just patch with the payload.

Share this post


Link to post
Share on other sites
  • 0
5 minutes ago, Forrest Evans - LM said:

Thanks for the feedback.  I'm confused why user logins would have such an impact on a collector, but I look forward to seeing your datasource.

We use User Profile Disks in a couple of our environments.  When a user logs in, it creates a symlink to a mounted VHD volume.  As this reads as the system adding a device, it registers as an instance under the volume dataSources.  When 2000 users log into a load balanced RDP Terminal Server set, that's a ton of instances that come online all at once.  One of the drawbacks is that the symlink doesn't disappear when the user logs out.  So every user's symlink is potentially on every server in the LBSet.  So 2K users across 10 Terminal servers = 20K potential devices that LM can't differentiate between a live link and a dead link.  So it ties up the resources for 18K instances while they wait to time out.  It also seems as though (when looking at !tlist c-wmi h=<IP for 1 TS>) that it's holding onto them even after it's timed out... so the socket pool never empties on the collector and a ping query with no sockets becomes a "host down" alert in LM for each host that isn't able to respond to the ping that is never happening.

We auto generate tickets based on those host down alerts, so we were bombarded with alerts the other day that would have been hundreds of false tickets that we'd need to pursue (our management uses those tickets as metrics as a basis for our performance as employees).  Luckily, I hadn't turned on the ticketing yet or we would have had a deluge like we haven't seen since the great AX-AOS alert storms of '17 ;)

Share this post


Link to post
Share on other sites
  • 0
12 minutes ago, Forrest Evans - LM said:

 If you are changing the collectorId within Auto Balanced Collectors there are some other "rules" to follow.  If you want to switch a device from being Auto Balanced to directly assigned you have to set `autoBalancedCollectorGroupId` to 0 and `preferredCollectorId` to the collector ID you want it directly assigned to.   To set it back to AutoBalanced from directly assigned do the reverse: `autoBalancedCollectorGroupId` sets to id of the group and `preferredCollectorId` to 0

With Version 2 of the API you don't need the $queryparam part.  You can just patch with the payload.

Interesting, preferredCollectorId points to an individual collector on every device in our environment currently.  Should they all be set to 0?  When we setup, it was before autobalancing was available, we switched the collector groups that we had to autobalancing afterwards.  What would the impact be of setting the group to 0, preferred collector to the new collector we want it to point to, once it moves, set the group back to the original?  Will it tend to shift back or will it be re-evaluated by the group and potentially shifted back even after I move it manually?

Share this post


Link to post
Share on other sites
  • 0

While you pass 0 to set it to AutoBalanced the actual collector that is currently doing the monitoring will be represented in the return value for 'preferredCollectorId'

I'd have to try and test your specific scenario and potentially confirm with the developers to see how it would respond.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.