I talk to a lot of MSPs about automation, and invariably we discuss our automation tools. It always surprises me when an MSP says, “we’re too small for that” or “I don’t think we could afford that!”. That usually leads to an interesting conversation, something like:

So – how many endpoints do you have? Around 600, including roughly 80 servers. That’s across 26 clients, and four of them are break-fix – we use the agents just for remote support.

What do you charge for basic support package – monitoring, patching, AV, AM, and maintenance?
 Charge? We only charge for the actual time we spend fixing stuff. We do charge $5 for the AV license.

Other answers range from “$25 for the basic package of patching and AV/AM" to "$150 for everything, including call-in support".

Well, with the initial cost of the VSA agent and the monthly costs for agent and AV/AM licensing, if you monitor and respond to alerts, you need to generate $250-400 per year on each agent just to break even. That means you need to charge at least $28 for just the basic monitoring and AV/AM, or find a way to reduce costs. (gulp!) I didn’t think of it that way.

How many monitoring alerts do you get each day? Too many (laughs). We used to get over 200 each day, so we turned off the monitors.

How many techs do you have on the help desk? Four – two L-1, an L-2, and an L-3.

How’s that working out – is their time fully utilized? It is – we could add another tech if we could afford it! The L-3 engineer is pretty busy with escalations and some manual maintenance tasks.

What’s your rough payroll for your four techs? Around $300,000, plus benefits and such.

What about customer projects – how do you handle those? We have one dedicated engineer for projects, and press the other techs into service  as time permits, but then the tickets pile up.

Do you do any ongoing system maintenance? We tried, but the procedures are difficult, don’t always work, and we’re too busy to spend more time on it.

Don’t you think that regular system maintenance might reduce the calls and alerts? It might, but that’s complicated to set up, and we just don’t have that expertise.

Well, I think that if you took a look at the results that our tools provide, you might change your mind. Here’s an example from an MSP that we first developed these tools for.

When we started working with them, they had around 1200 managed agents (only 16 were break-fix). They had two L-2 and three L-3 engineers on the helpdesk, with base tech salaries hitting just over $500,000. Another two engineers working on projects pushed tech salaries to $720,000. Those endpoints generated over 200 alerts per day – that’s one in six machines generating alerts through monitoring. There was one engineer per 240 endpoints, so each endpoint had an “employee cost” of $421.

During the first year, we developed standards that helped streamline their operations, we removed the sample monitors and deployed ones that were engineered to their customer requirements, and added some basic automation to VSA to deploy monitors based on detected services and schedule patching and updating. This is what we now sell as our Core Automation Suite.

After the $1500 investment in this solution, they were able to increase their managed endpoints to just over 1800. They eliminated the L-2 helpdesk positions and promoted “one call and done” help-desk services to the clients using only L-3 engineers. Alerts dropped to 75 per day – one in twenty-four. With one engineer per 600 endpoints, the “employee cost” of an endpoint was now down to $192 – a savings per endpoint of $229.

The following year, we helped them implement the EMM Suite, which includes Smart Monitors and Daily Maintenance. The Smart Monitors reduced alerts by auto-remediating many common conditions and dynamically setting reasonable alert thresholds. Daily Maintenance improved the operation of the endpoints and further reduced the help-desk load. By now, they had grown to almost 3000 managed endpoints, yet alerts dropped to under 16 per day (one in every 188 agents)! Customer calls had also dropped since they were proactively maintaining the systems. One help-desk engineer now supported almost 1000 endpoints, dropping the “employee cost” of an endpoint to just $110 per year. After a year using the EMM suite, they reported that the help desk team spent nearly 50% less time on tickets because the basic remediation tasks were completed through automation.

Thus – a $1500 investment in automation allowed the elimination of two L-2 help desk seats, freeing those employees for project work and still supported a 50% increase in managed endpoints. Using our automation for patching and daily maintenance, end-user calls dropped, and customer satisfaction increased. The $0.50 cost for these improvements was easily absorbed by the increased profitability, and our end-user interface that reported what was being done eliminated the “I pay you all this money – what do you do for me?” question that so many customers ask! We tell them every day what we’ve done.

Here’s some facts based on clients that we’ve helped.

Typical “small” MSP has around 750 managed endpoints, and a few hundred “break-fix” systems. They have 3 techs on help desk at L-1 to L-3, plus another 2-3 on staff for project work.

Without automation:

• 3 Help Desk techs base salaries (major city region) cost $250,000
• About 125 alerts per day are generated – 42 per tech, and hard to review and close all of them.
• Each managed endpoint requires $334 per year to break even on salaries alone.

With automation and maintenance:

• 2 Help Desk techs (L-2 and L-3) base salaries are $195,000. $55,000 per year savings.
• $4,500 per year in EMM licensing, still a $50,000+ annual savings (not counting that the 3rd tech can now be billing for project work with no additional salary cost!)
• Fewer than 5 alerts per day – 2-3 per tech, easily handled, allows direct support of client calls.
• Each managed endpoint requires $266 per year to break even on salaries and the $6 EMM license costs. (Yes, EMM costs just $6 per agent per year.)
• At the industry average of 500 agents per help-desk agent, the break-even cost is around $150, and we’ve seen 1100-1200 endpoints per agent possible once the environment has been patched and maintained for a few months.

The question is – what’s more expensive? $3000 plus $6 per agent per year or the salary of another tech? Plus benefits, payroll taxes, insurance… Other considerations include:

• What about your employee satisfaction vs. the frustration of fighting a losing battle against alerts?
• Then there’s client satisfaction – their network gets “quiet”, the “fires” stop, and their employees become more productive.
• Can you do this yourself? Sure, but at what cost? Can you dedicate an engineer to build the monitors and the tools and then maintain it? Or will they get pulled into customer support?
• Then there’s the extra billing capacity for new work by employees that aren’t tied to the help desk!

NOTE: Salary costs were based on averages in major metropolitan areas like NYC, Boston, and LA. Level 1 tech salary is $55,000, Level 2 is $80,000, and Level 3 is $110,000. EMM costs are based on an average distribution of 12% servers and 88% workstations.