The Dedicated MVP NOC: The Simplest Way to Achieve Next-Gen NOC Support

MVP NOC
Prasad Ravi

By Prasad Ravi

President, INOCPrasad has more than 25 years of networking and IT experience. Prior to INOC, he was Director of Enterprise Network Services at Rush University Medical Center. Previously, he applied computational science methods to problems in engineering at the National Center for Supercomputing Applications. Prasad holds a Ph.D. in computational science from the University of Illinois at Chicago and an MBA from Northwestern University. He is the author of 14 published papers.

It’s a question we hear all the time: “What’s the simplest, most cost-effective way to turn up 24x7 NOC support on INOC’s framework/platform with a team dedicated to us?”

Framed slightly differently: “How can you help us free our internal staff from break/fix work while improving performance against our service levels in a way that’s most cost-conscious?”

Here, we answer these questions by introducing and briefly explaining what we call the Dedicated MVP NOC.

In short, the Dedicated MVP NOC is the simplest, least expensive way to turn up 24x7 NOC support on the INOC Platform and reap the benefits of INOC’s operational framework (i.e., unburdening the internal support team from frontline monitoring and management activities and improving uptime and availability across the supported environment.) 

Companies that engage INOC for 24x7 NOC service typically reduce their high-tier support activities by 60% or more, sometimes as much as 90% over time. Results like these are possible thanks to the operational framework we’ve refined for over 20 years and the platform we use to deploy it with our NOC clients.

Many challenges signal the need for an external 24x7 NOC support solution that includes these operational and platform capabilities.

Here are just a few of the most common:

  • Your advanced engineers (Tier 2 and 3) are spending too much time addressing basic (Tier 1) support needs.

  • You have little or no training in place to document and transfer knowledge across your support team—or to new hires.

  • You don’t have a support platform that allows you to capture enough good data to analyze your issues and uncover trends you can act on. (Quick check: can you name the top five sources of your incidents over the past six months, and how many hours you devote to them each month?)

  • You’re preparing to launch or expand a new product or service and don’t have the time or budget to scale your monitoring and management operation to support more customers.

  • Your leadership, customers, or end-users are unhappy with the level of ITSM they’re currently receiving—and there’s no lever to pull internally to change that as rapidly or cost-effectively as you need to.

If you don’t have the budget or support utilization to justify a more comprehensive NOC support operation, the Dedicated MVP NOC offers a way to receive the core benefits of one for a fraction of the cost and complexity.
Prasad Ravi, CEO, INOC

“So much of the support work that ties up internal technical teams can be resolved by a highly-trained Tier 1 team armed with great runbooks and a NOC support platform that makes their actions efficient and consistent. With the Dedicated MVP NOC, we can spin up a 24x7 NOC service so teams can get their time back.”

— Prasad Ravi, CEO, INOC

What is a "Dedicated MVP NOC"?

A Dedicated MVP NOC is the bare minimum needed to stand up and operate a 24x7 network operations center to perform its most basic functions.

While the “minimally viable” NOC as we describe it here offers a comparatively quick and cost-effective way to turn up 24x7 NOC support, it’s only equipped to support the infrastructure as it exists; it’s not equipped to bring new assets (network components, circuits, infrastructure, applications, etc.) into the supported environment.

Two main groups will find this model attractive:

  • Teams that need a blueprint for their own NOC if they want to build one internally (see our blog post “Outsourced NOC vs. In-House NOC”).

  • Teams that want to turn up outsourced NOC support with INOC to improve uptime and availability while freeing their internal resources from support tasks as quickly and cost-effectively as possible.

Note: In addition to Dedicated NOC support solutions (e.g. Dedicated MVP NOC), INOC provides other options for 24x7 NOC support: Shared, Hybrid, and Designated NOCs.

What Does a Dedicated MVP NOC Include?

In short, the simplest 24x7 NOC is comprised of ten full-time employees responsible for three core tasks:

  • Event Management
  • Incident Management
  • Change Management (specifically, Scheduled Maintenance Support)

Why ten people? Because running a 24x7 NOC requires staffing three shifts a day, 365 days a year. Factoring in things like PTO, we’ve found that ten is the absolute minimum number of staff required to operate a round-the-clock NOC with as little “slack” as possible to reasonably ensure coverage at all times.

Why Event, Incident, and Change Management? Because managing events and incidents are the core functions of the NOC. Scheduled Maintenance Support—a component of Change Management—is necessary to suppress those processes during scheduled maintenance windows. Without it, the NOC would receive false alarms that arise due to planned downtime. 

The easiest way to dig deeper into the MVP NOC is by describing the three essential elements that comprise it: people, process, and platform.

People

Let’s briefly break down that ten-person Dedicated MVP NOC team.

Usually, such a team is composed of eight Tier 1 staff with one Tier 2 engineer and one manager. 

The actual composition of any specific MVP NOC may require more than this, of course. For example, a company may find that, based on its alarm flows and SLAs, it actually needs three Tier 1 engineers during a certain shift. The staffing plan would be adjusted accordingly.

Note: Staggering Tier 1 NOC staff allows for smoother incident hand-offs between shifts. However, a well-defined Incident Management process is important for quality control and operational management. In most cases, the Tier 2 resource(s) and manager should be scheduled to work during standard business hours to allow for timely interaction with company staff from other departments.

Below is a high-level overview of each level within the NOC organization. Again, like everything described here, these are subject to change based on the factors at hand.

  • Notification Support is responsible for detecting and identifying events from alarms, calls, and emails related to things like network and security equipment, circuits, cloud infrastructure, and applications. This also might include creating incidents and notifying or escalating them to the client or customer until they’ve been acknowledged.
  • Tier 1 Support is responsible for initial event correlation, infrastructure and service impact determination, and incident prioritization within established SLA timeframes (in addition to the scope of Notification Support). These engineers should have fundamental expertise in network and IT technologies. Many times, Tier 1 may also need expertise working with multiple third-party companies to manage the service restoration process. We typically recommend Tier 1 resources be utilized 25% of the time for Event Management, 60% for Incident Management, 10% for training, and 5% for administrative work.
  • Tier 2 Support expands on the Tier 1 expertise to include a deeper, more investigative level of troubleshooting expertise in network and IT technologies. Specific competencies are subject to the particular technology environment being supported.

The table below goes one step deeper, outlining a generic breakdown of duties and responsibilities at each level. In most cases, Notification and Tier 1 support are both provided by the Tier 1 staff.

Notification Support

  • Detecting and identifying events from alarms, calls, and emails related to network and security equipment, circuits, cloud infrastructure, and applications

  • Creating incident tickets

  • Notifying and escalating with customers or end-users until incidents have been acknowledged

  • Searching knowledge databases

  • Coordinating support activity with customer or third-party organizations and field personnel

Tier 1 Support

  • Conducting initial event correlation, infrastructure and service impact determination, and incident prioritization within established SLA timeframes, in addition to the scope of Notification Support

  • Using equipment command-line interfaces (CLIs), EMSs, and web interfaces at a low-risk level

  • Using NMS interfaces to determine initial fault domain

  • Searching Known Error and knowledge databases for incident solutions and workarounds

  • Using technical scripts and procedures to perform initial diagnosis and attempt to restore service

  • Coordinating support activity with customer or third-party organizations and field personnel

Tier 2 Support

  • Using equipment, service, and infrastructure-specific knowledge at an intermediate level to investigate incidents and make diagnoses

  • Identifying and correcting configuration and interoperability issues suspected to be the root cause of incidents

  • Coordinating support activity with customer or third-party field personnel to replace faulty equipment and restore configuration and services

  • Validation/acceptance of service restoration efforts to ensure residual issues have cleared

  • Ability to use higher-level authorization accounts for interrogating or applying changes to infrastructure

  • More advanced comprehension of infrastructure-wide issues and impacts

  • Guiding/mentoring Tier 1 personnel in problem analysis and decision-making efforts

 

A word about training

To ensure advanced engineering resources aren’t overutilized for support work, the Tier 1 team should eventually be expected to resolve most incidents. Here at INOC, our typical goal at the Tier 1 level is 60% to 80% frontline resolution.

Achieving a high frontline resolution target like this requires robust training for the team—both initially and on an ongoing basis. That almost always includes incorporating foundational technical skills and certifications, ​​creating and testing against a competency matrix with technical skills, and establishing a mentoring program to transfer knowledge from other departments.

📄 Read our free white paper—Empowering the IT Service Manager—for a closer look at how to operationalize a NOC to significantly reduce the burden of support on advanced technical resources.

Process

While the Dedicated MVP NOC only performs three processes, the procedures that fall within them need to be highly efficient to make the best use of very limited resources. The higher the volume of support activity, the higher the stakes on efficiency.

While we can’t outline every single procedure a Dedicated MVP NOC would need to have documented and trained on, here’s a high-level breakdown of what specific roles and responsibilities fall under each process:

Event Management

  • Monitoring specific infrastructure
  • Generating alerts when any component of the infrastructure fails or exceeds defined operational parameters
  • Evaluating, filtering, correlating, prioritizing, and categorizing alerts based on an understanding of the supported infrastructure and provided runbooks

Incident Management

  • Providing first-tier Incident response, including creation, assignment, and tracking of incident response tickets as appropriate based on generated alerts or other notifications from the company’s staff or third parties; additional notification mechanisms can include calls and emails
  • Providing notification of outage events or other service impacts to relevant company staff
  • Providing online-accessible and real-time status updates for ongoing incidents with appropriate information
  • Escalating unresolved issues to appropriate staff
  • Closing incident response tickets upon service restoration, including timely notification to affected parties as appropriate
  • Managing major incident communications according to a set runbook and as requested, participating in post-incident root cause analysis and process improvement determinations conducted internally
  • Providing a daily status report of Incident Management status to operational management staff

Change Management (Scheduled Maintenance Support)

  • Scheduled Maintenance Support records and manages maintenance incidents that affect infrastructure and services. Maintenance Incident identification is done by email, call, or portal (communication from supported departments and third parties)

Note: AIOps can reduce and possibly eliminate Event Management activity and instead allow the NOC team to focus more on Incident Management. Download our whitepaper, The Role of AIOps in Enhancing NOC Support, for ideas.

Platform

There are three primary types of tools required to run the most basic NOC operation and report on its performance (however, specific support requirements may require additional tools be used):

  • An ITSM/ITOM tool (such as ServiceNow)
  • An NMS tool (such as SolarWinds)
  • A reporting tool (such as ServiceNow, Tableau, or PowerBI)

📄 Get expert tips for operationalizing tools in the NOC: NOC Tools and Software: An Operational Perspective

With an MVP NOC specifically, it’s important to select tools that offer more versatility and can be configured to do more with less. ServiceNow is a prime example of this level of versatility, as it can be used for ITOM, ITSM, and reporting if desired. Additionally, ServiceNow’s Service Portal can be provided to supported departments for ticket visibility, tracking, updates, and requests. 

To illustrate its simplicity, a sample event and ticket flow with this configuration might look like this:

Email Alerts/SolarWinds  →  ServiceNow ITOM  →  ServiceNow ITSM  →  ServiceNow Reporting

While tool selection is critical in designing an MVP NOC, the way those tools are configured and integrated is equally important to ensure the smallest possible NOC team can still work efficiently to meet SLAs while preventing the stress (and eventual burnout) that results from constant operational frustrations.

Over the 20+ years INOC’s been refining NOC operations, our team has developed a unique playbook of platform configurations and integrations that enable our Dedicated MVP NOCs to operate with minimal resources. These best practices speak to our core competency and often require (or are at least greatly enabled by) the INOC Platform.

This includes, for example, configuring the ServiceNow ITSM as a “single pane of glass” for the NOC team by:

  • Incorporating specific services like Incident and Scheduled Maintenance Support as Services with associated Service Offerings within ServiceNow’s Service Portfolio Management.

  • Implementing a comprehensive Service Level Management framework for the MVP NOC based on target SLAs and OLAs.

  • Automating ticketing utilizing the Event Management, Configuration Management Database (CMDB), Alert Correlation, and Flow capabilities of ServiceNow.

  • Configuring ITOM to process inbound email alerts and create tickets in ITSM based on alert criteria.

  • etc.

We also employ “standardized” best practices to configure workflow management by:

  • Implementing Flows in ServiceNow to automate runbooks.

  • Building templates in ServiceNow to standardize field completion.

  • Implementing an escalation workflow (for example, to govern how cases should be “returned” to the MVP NOC or Owner when pre-determined elapsed times are exceeded without a ticket update and/or status change).

  • etc.

Effective and complete runbooks are critical for keeping a Dedicated MVP NOC efficient. These operations also need documented procedures for tool usage, Scheduled Maintenances, shift hand-offs, and on-call/department escalation contacts.

Reporting is one area most NOCs in general struggle with. What reporting exists is often solely focused on performance-based KPIs; resource utilization tracking is just as important—especially in the context of a Dedicated MVP NOC.

Whether using a multi-use tool like ServiceNow or a tool like Tableau, reporting should be configured to:

  • Establish a baseline of performance that can be tracked over time with metrics that speak to alert handling, average edit time, frontline resolution rate, etc.

  • Enable the ITSM tool to measure the number of edits and the labor content per edit for each engineer.

  • Surface chronic alarms with a weekly or monthly review for top trending issues.

  • etc. 


Service Level Management

As mentioned earlier, every NOC should have clear guidance for measuring service levels to ensure its holding up its commitment to the organization or customers it serves. 

Each core task of the NOC that’s assigned an SLA should be further broken down into discrete Service Level Objectives (SLOs) and Service Level Indicators (SLIs) with time targets and descriptions that clearly articulate the expectation for service.

📄 Read our in-depth guide to SLAs and SLM for more: NOC Service Level Agreements: A Guide to Service Level Management

Access INOC’s Support Framework & Ops Platform for a Fraction of the Cost and Complexity

Want to learn more about our Dedicated MVP NOC, and how INOC can help you hit your uptime and performance targets while saving your team valuable time and resources? Use our contact form or request a free NOC consultation to tell us a little about yourself, your infrastructure, and your challenges. We'll follow up to better understand your goals and challenges, and explore possible NOC solutions.

Want to explore the benefits of strategically outsourcing your NOC support? Read our definitive guide to outsourcing and grab our free white paper below, which identifies the top 10 challenges NOCs face with practical advice for solving all of them. 

Top 10 Challenges to Running a Successful NOC

FREE WHITE PAPER

Top 10 Challenges to Running a Successful NOC — and How to Solve Them

Download our free white paper and learn how to overcome the top challenges in running a successful NOC.

Download

Prasad Ravi

Author Bio

Prasad Ravi

President, INOCPrasad has more than 25 years of networking and IT experience. Prior to INOC, he was Director of Enterprise Network Services at Rush University Medical Center. Previously, he applied computational science methods to problems in engineering at the National Center for Supercomputing Applications. Prasad holds a Ph.D. in computational science from the University of Illinois at Chicago and an MBA from Northwestern University. He is the author of 14 published papers.

Let’s Talk NOC

Use the form below to drop us a line. We'll follow up within one business day.

men shaking hands after making a deal