Eric Singer

Read only agent / collector

Recommended Posts

I know I've brought this up before, but I'd like to bring it up again.  LM's requirement that collectors run as local admins (or system) is a GAPING security hole in your product.  No amount of certificate signing, or other like security measures are a replacement for running a collector or an agent as a read only account.  The fact is, with every security measure you take, if the collector is running as an admin account or a system account, its going to be exploitable in one way or another.  Having the signed scripts and what not, would be great, but really it shouldn't be the primary focus IMO.  Security is much better when its locked down by default and opened up as needed, compared to what you guys are doing, which is a completely open system, that you're trying to add security enhancements on top of.  It's almost akin to you guys having no firewall,and then adding a few rules here an there to block certain types of traffic, while the rest of the network is completely exposed.

A more prefered architecture (security wise) would be an agent / collector that can run as a read only account and be supported.  WMI, perfmon, and many other functions all work fine with a regular user, when it's executed locally.  That is why an agent or a special collector is needed.  Most ideal communication path would be an "agent" talks to a "collector" which then talks to the portal.  This would also allow us to keep our internet locked down.  I suspect this would also have the other advantage of taking a lot of load off the collectors and really putting most of the work on the agent, which is ultimately better given that the workload would be distributed.

For now though, even having a "supported" configuration for a collector not running as a local admin / system would be a great step in the right direction.

The reason this is less of a concern for solution like Solarwinds and SCOM is they're on premises based solutions, meaning there is much lower external risk factor.  You guys are cloud, and there for need to design the solution from an untrusted point of view.  

  • Upvote 1

Share this post


Link to post
Share on other sites

Hi Eric -

LogicMonitor supports running your Windows Collectors however you like. The only requirement -- imposed by Windows -- is that it have sufficient permissions to poll WMI data on the devices to which it's assigned. To do so, the Collector needs to either be running as an account with permission to access WMI services on the target hosts, or be provided with credentials of an account with such access. Although many of our customers prefer to run their Windows Collectors as a local admin or system account, LogicMonitor does not require that they're configured in this way.

If you prefer to install a Collector in a per-host "agent" model, that's absolutely fine with us. In fact, we've provided APIs that make it easy for you to do so using automation/orchestration tools. See https://www.logicmonitor.com/support/rest-api-developers-guide/collectors/add-a-collector/ for details.

Regardless of your preferred Collector deployment model, as a security best-practice we recommend the use of role-based access control to limit access to Collector  and LogicModule Management to a minimal number of individuals. This ensure that only those you've authorized have remote access to your Collector environment.

Share this post


Link to post
Share on other sites

While I appreciate the security points raised by Eric, I would have to reconsider LogicMonitor if I had to go back to installing maintaining agents across our estate. 

In my estate, we use a dedicated account for WMI for Windows, Linux and network monitored via SNMP and ESX using dedicated account for ESX.  For custom data sources and event sources for applications and databases use we dedicated accounts also.  Our production, dev & test VLANs and DMZ are firewalled.

Edited by Mosh

Share this post


Link to post
Share on other sites

@Matthew Dunham  I haven't had any luck finding a KB or how-to on running an LM collector as a non-admin account.  The few articles I have found here https://www.logicmonitor.com/support/getting-started/i-just-signed-up-for-logicmonitor-now-what/3-adding-collectors/ and here https://www.logicmonitor.com/support/getting-started/advanced-logicmonitor-setup/running-without-administrator-privileges-in-windows/ only discuss setting up a collector with admin rights or stating "not supported and not reccomended" in the verbiage.  So if you could point me to a doc, that show how to run as a non-admin, with *supported* in the verbiage, I'll be happy to give it a go.

In regards to this comment " Although many of our customers prefer to run their Windows Collectors as a local admin or system account " I get that.  I like things that are really easy to.  But the easy way and the right way are two different thing.  There was a time when everyone used to run as a local admin on their desktops (easy way).  Then we realized that was dumb, and security conscious companies started running as non-local admin accounts (right way).  You shouldn't let customer preference determine proper security.  it's one thing if you provide a *good* option to run your solution in a secure manner and the customer ignores it.  It's totally different if you don't provide a good option and basically make the customer jump through a million hoops to lock down the gaping holes in your solution.  You guys IMO fall into the latter.  You suggest we run an RBAC all the while having a solution that's founded on local admin rights.  Those two don't go together.  RBAC is based on a principle that you start out in a 100% locked down configuration, and only delegate the minimum needed rights.

I acknowledge that Microsoft has created a pretty terrible solution for remote polling, but it is what it is and its not going to change.  Centrally managing WMI + DCOM rights TMK doesn't exist, at least not easily.  Centrally managing MSI (agents) though is relatively speaking a trivial task.  I don't love the idea of having to update 700+ devices every time there's a new update, but comparing that to having a solution that's one exploit or hack away from destroying my environment, I'll take it.  You need to realize how dangerous your solution is to windows environments, and frankly your customers should really start pondering it.  One person hacks and admin login, and that customer is toast.  Create an active discovery script with a single line of "shutdown -f -t 0" and every windows system being monitor will shutdown until the collector shuts its self down.  or even worse, deploy an active discovery script that downloads malware or some other dangerous thing and again, goes to down.  You can come back and say signing will solve all of this, but it doesn't really.  It reduces the vector, but it doesn't eliminated it.  There's always a chance there's an exploit in the JVM you use, which would allow someone to bypass the signing.  Having that JVM (or whatever agent service tech) run as an account that's not a local admin would mitigate most if not all of those issues.   Sure, there is still a chance there's an exploit, but at that point, its on MS and not you guys, which is honestly what you should want.

In general though, I'm not asking that LM change its architecture so that it ONLY works the way I requested, I'm asking for an enhancement for those of us that actually care about security in our windows environment.  I'm going to make a pretty bold statement, and say that if you're ok with running LM collectors under a service account that has full admin rights across every server it polls, that you don't care about security.  IMO, running full blown collectors on every system is a hack, and not a good long term solution.  You guys really need an agent, and I get that maybe it means a lot of changes on your end.  However, its the right solution long term for a secure windows environment.  

You guys make a kick butt monitoring solution, so I'm not trying to be disparaging, but this is a huge issue IMO.  I've brought this up several times and each time its either its "too hard" or its dismissed as a non-issue.  I'm glad you've added many of my requested features, but I think this is one that deserves some serious consideration.  I would love to chat more in person about ideas if you guys do decided to pursue it further.  I have a few ideas on how I'd love to see it implemented.

@Mosh  I get what you're saying, but it's the only way Windows will let you easily poll data without admin credentials.  Going agentless + no admin rights is WAY harder to deal with than installing an agent + no admin rights.  

 

Share this post


Link to post
Share on other sites

For me it's not the updating of the agent software that is the most costly element, it's the months and months of raising change requests, getting the approvals and negotiating change windows for thousands of machines.  We would probably never be able to stay up to date because by the time we've been through one update cycle, it would be time to start planning the next update.  This is why we got rid of our previous agent based solution. Sure, we can automate the updates, but not the human element of business and IT operations.

In an ideal world, yes we'd have the resources to do everything, but in practice we have to balance risks.  So, what we're doing is to focus on access controls across the estate and security audits/alerts.  We're comfortable with the security of our data centers; the primary surface area for malware in our environment is the end user compute space, but even so, regular users cannot perform WMI queries or issue commands to Windows servers, and the user VLANs and data center VLANs are firewalled.  We limit the number of LM admins to a trusted few (just two people).


 

Edited by Mosh

Share this post


Link to post
Share on other sites

@mosh

That's fine for you guys, but for us its not.  I'm not suggesting that LM eliminate agentless monitoring as an option.  It really shouldn't be that hard to develop an agent or a lightweight collector that is specifically designed to run as a non-admin account.  

Share this post


Link to post
Share on other sites

I would also like to add that if we're going to need a full blown collector, that I think even a "nano" sized collector is overly large for single device monitoring.  I'd ideally like to see something in the 512MB or less memory wise and 1 vCPU.  if all its doing is monitoring locally, I would like to assume it doesn't need a ton of resources.  Of course that a big assumption.  Admittedly, 512MB is an arbitrary number I made up, but something less than 2GB for sure.

Share this post


Link to post
Share on other sites
On 2/16/2017 at 9:32 PM, Matthew Dunham said:

Hey @Eric Singer -

My apologies -- I didn't realize the state of our documentation for this use-case. I'm working on rewriting, and making sure our Support Engineers are in a position to support this use-case.

 

Matthew (or other LM'er):   looking at the docs as they stand now, it would seem that to use WMI without administrator-level priveledges the guidance is that each monitored windows node must have local security policy changes.   Is that correct?   Is there any AD-only solution where the service account in use for WMI polling has the minimally required rights to accomplish the basic goals?   Is it further complicated when domain controllers are used as the systems running the collector (understand that using domain controllers for collectors is not the best choice, but as an MSP I can only recommend against this and in some cases our hand has been forced).  Specifically, we need an account that can poll WMI but cannot execute powershell or other scripts that can modify the environment.
 

Share this post


Link to post
Share on other sites

We've done some in-house research into the right-sizing of the appropriate permissions for polling WMI/PDH from a remote system. Our own experience has been that we can indeed craft a set of non-administrative permissions carved out for this use-case for a given OS version + service-pack + patch-level. But that a subsequent patch may change these on any particular Patch Tuesday, which makes it pretty challenging for us to support -- or even recommend. 

We have an initiative in-flight with Microsoft Consulting by which we're attempting to get direct guidance from the proverbial "horse's mouth" on how best to right size these permissions in a future-proof (or at least future-resistance) way. Stay tuned....

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.