Chapter 34 - IT Contract Administration

34.2.5 Monitor supplier performance

The contract administrator should carefully review the contract for all supplier performance requirements; some of which are clearly defined and others which may be indirectly included. It is recommended that the contract administrator develop a performance checklist and calendar to ensure all performance requirements are exposed for scheduled monitoring.

The contract may specify a clearly defined service level agreement (SLA) or acceptance criteria for deliverables describing expectations for the supplier's technical performance. The contract may also necessitate that indirect supplier performance requirements be monitored-the submission of recurring technical, status and/or administrative reports (see subsection 34.2.2); compliance and licensing certifications (see subsection 34.2.2); key personnel, and business or project management obligations.

• Technical performance-Needless to say, there are many variations on technical performance objectives and requirements that could be in a contract. The contract administrator must know exactly what these are and what performance data capture, measurement and reporting requirements exist in order to monitor the supplier's or product's performance and determine how the performance results may impact other areas of the contract's administration, including supplier payments. Close teamwork between the contract administrator and other contract stakeholders will ensure that all required performance results are captured, reported and used in monitoring a supplier's or product's technical performance. Here are examples of how a supplier's or product's technical performance can be captured and monitored.
  • Service level agreements will often require that the supplier provide monthly performance reports that indicate how well they met the performance requirements in order that related monthly invoices reflect any non-performance penalties/incentives (usually tied to a percentage of service level attainment or non-conformance). Such performance parameters may be linked, for example, to "uptime" and "downtime" of a system or web-hosted application or to "response time" and "remedy time" for the procured service.
  • Acceptance criteria may require, for instance, that 100 simultaneous users of a system will not degrade system performance or impact the required maximum time it takes for a user to complete a certain electronic "transaction" or "function." If the system's built-in self-monitoring design captures degradation or error capture then reports can be generated by the agency's system administrator and reported to contract stakeholders.

• Schedule and budget-Schedule and budget are two risk-critical performance areas common to non-IT as well as IT contracts. Schedule and budget performance have the potential to severely impact the agency's funding sources-Commonwealth and/or federal grant-and dependent planned or existing technology projects, as well as the supplier's financial condition.

  Here's a hypothetical example: A contract for developing and implementing a replacement for a critical incident tracking system that links into multiple statewide databases and a federal database was awarded by a Commonwealth agency on January 1, 2008. The application software of the current 10-year old system will no longer be supportable or operational after June 30, 2009. The contract includes a major project milestone deliverable for a 60-day acceptance testing to be complete February 1, 2009. The total contract fixed price is $3 million dollars-half from General Funds and half from a federal matching grant. Thirty days into the acceptance testing a major error occurs that the supplier cannot fix without redesigning a critical interface. By this point in time in the project, due to payments triggered by the completion of interim milestones, $2.5 million of the project budget has been paid to the supplier. With only 6 months and only $500,000.00 in funds remaining before the old system goes away-"Houston, we have a problem."

  You can see how budget and schedule performance have the potential to impact both immediate and extended stakeholders. We'll never know how this project turned out and who suffered the biggest impact, but more than likely it ended up in litigation.

  Monitoring schedule and budget performance areas may be assigned to the contract administrator, the business owner/project manager or the contract manager, depending on the size and complexity of the procurement and the level of project management required. If the contract required CIO approval, then VITA's Project Management Division has ProSight and other tools that the contract's project manager will probably use in performing contract administration functions on these two performance areas. However, the contract administrator will need to be knowledgeable of communications, problems and resolutions surrounding these performance areas and be involved in any related fact-finding, communications, modifications, disputes or claims.

• Key personnel-Often an IT contract will include a key personnel term similar to this: "The statement of work may designate certain of supplier's personnel as key personnel or project managers. Supplier's obligations with respect to key personnel and project managers shall be described in the applicable statement of work. Failure of supplier to perform in accordance with such obligations may be deemed a default of this contract or of the applicable statement of work." Additionally, a contract may require written approval by the agency if supplier desires to replace key personnel or named individuals. It is important that the contract coordinator work closely with the business owner/project manager to monitor and enforce whatever terms are in the contract.

• Project/business process management-The contract administrator should search the contract and exhibits for any performance obligations related to project/business process management. These may include relevant deliverables; i.e., project plans, quality assurance plans/reporting, project status reports, cutover plans or specific project management and operational processes that supplier is committed to follow and/or substantiate either by providing a defined deliverable or simply by operation of business (i.e., using standards IEEE 1220 and ISO 1006 for monitoring project planning, etc.).

  The supplier's contractual performance must be measured by all performance elements and criteria established in the contract. While the reporting, collection, monitoring and evaluation of supplier performance data may be a collective effort by other contract stakeholders, the contract administration function should act as a repository for all performance data and act as overseer to ensure that contractual performance requirements are monitored and reported.

  If a supplier's performance is unsatisfactory, the contract administrator and other contractual stakeholders should document, with supporting evidence, their complaint of unsatisfactory or non-conforming performance. For all VITA-issued or VITA-delegated IT contracts, contact scminfo@vita.virginia.gov. For all non-VITA issued or agency non-delegated IT contracts, contact DPS Contract Compliance.