Emergency Patch Management

Windows Systems

NICE CXone must apply patches to the affected CXone systems as soon as possible, if:

  • Proof-of-concept code is publicly available regarding a possible exploit.

  • A new critical security patch is released.

The purpose of these patches is to immediately remove vulnerabilities in the organization's cloud environment. Microsoft Critical Updates and other patches are run through a test cycle and applied on a regular schedule. This schedule is typically a 30-day patch cycle beginning with Microsoft's Patch Tuesday. NICE CXone also maintains a Linux patching schedule with a similar priority rating. Critical patches should not wait for the next month's patching.

Patches and upgrades are documented in a Change Control Record (CCR). They should be reviewed by the NICE CXone Change Management Board for the following:

  • Impact.

  • Risk analysis.

  • Necessary actions taken before approval.

Patches and upgrades should also go through peer and technical reviews.

Patches and upgrades are first tested in the NICE CXone lab, then in alpha and beta clusters, then in production. NICE CXone maintains an emergency change management process for unexpected critical issues.

Testing is performed on updates by a dedicated Quality Assurance (QA) department. This department utilizes practices within NICE CXone guidelines. These guidelines are found in the following:

  • Security Development Life Cycle (SDLC).

  • Policy and procedure documents.

The NICE CXone operations teams update or patch all applicable CXone devices. Patching takes place during a set maintenance window, normally covering three off-peak hours nightly, depending on the location. The maintenance window is disclosed as part of the customer's contract. Maintenance can result in a brief interruption while servers are rebooted, or clusters are failed over.

Non-Windows Systems and Software

Linux and other non-Windows operating systems and devices are patched on a calendared and ad-hoc basis. They follow the same Change Control Board (CCB) process and cycle as above. Current versions are known and compared to relevant common vulnerabilities and exposures (CVEs), then remediated.

Information Security evaluates third-party or non-Windows software. They do so using:

  • QA-driven scanning.

  • The Security Center.

  • Other detective controls and reports.

Information Security communicates these vulnerabilities to respective owners and groups, such as SysOps or R&D. Vulnerabilities ranked "Critical" or "High" should be patched within 30 to 90 days. Certain threats within those vulnerabilities should be managed within the 30-day mandate.

Tools

Infra-Tools is an internal tool used in Windows. It performs automated software and hardware status discovery. This tool is run by System Operations groups. It ensures that unpatched machines and software are flagged and patched.

Firewalls do deep-packet inspection during IDS/IPS with threat protection enabled.

NICE CXone employs other administrative threat landscape survey tools internally. Logging SIEMs inspect logs from appropriate servers for anomalies.

Service-Level Objectives

NICE CXone uses service-level objectives when applying patches. These objectives ensure that:

  • known vulnerabilities are patched on time.

  • patches are compliant with:

    • PCI-DSS

    • Other scoped security infrastructure requirements.

A scale is used to describe patch and event risk. The internal device owner group rates the venerability as "Critical," "High," "Medium," or "Low."

  • Critical vulnerabilities must be patched within 30 days.

  • High vulnerabilities must be patched within 60 days.

  • Medium vulnerabilities must be patched within 90 days.

  • Low vulnerabilities must be patched within a time set by the internal device owner group.

Information Security passes on information about weaknesses to the Security Center. It does so with vulnerability inspection reports.

Remediation Cycle

The analysis for critical threats begins immediately upon discovery. The threat definition and plan of action are determined within a 72-hour period. Then the plan goes into the remediation cycle. The remediation cycle consists of the following:

  1. Architecting a solution.

  2. Engineering the solution.

  3. Implementation in the lab for testing.

  4. Passing the solution on to QA.

  5. Release scheduling.

  6. Change Board peer review and approval process.

  7. Deployment.

The analysis, definition, plan of action, and remediation cycle must occur within 30 days of detection. If a tenant service is impacted, the tenant is notified of progress updates. Updates are delivered as each step of the remediation cycle is implemented. After successful remediation, depending on the incident, an RCA is delivered to impacted tenants.

Exceptions

Manual remediation and cataloging is performed for servers and devices that are not:

  • Communicating with WSUS servers.

  • Able to pull updates.

Reports of unpatched servers are provided to Information Security for visibility, tracking, and remediation actions.