Technology Portfolio Management (TPM) Best Practices

Overview

AI technologies create governance challenges that no other technology category in the portfolio presents in the same form or at the same intensity. The most significant of these challenges is the non-static character of AI behavior: unlike conventional software whose behavior is determined by its code and changes only when its code changes, AI systems whose behavior is determined by their underlying models can change behavior as models are updated, as training data changes, or as the distribution of inputs they receive in production diverges from the distribution on which they were trained. Governing a static artifact is fundamentally different from governing a continuously evolving behavioral system, and the standard governance disciplines designed for static software must be adapted for the AI context.

The regulatory environment for AI governance has moved rapidly from principles-based guidance to enforceable requirements. The EU AI Act establishes a risk-tiered regulatory framework for AI systems that imposes specific transparency, accountability, and performance requirements on AI systems classified as high-risk. The EU AI Act and comparable frameworks in other jurisdictions are creating a compliance landscape for AI technologies that is as consequential as the compliance landscape for financial data protection or health data — and organizations that have not begun building AI governance capabilities are already behind the compliance timeline. (Source: EU AI Act, Regulation 2024/1689.)

Best Practice

Address the unique governance challenges of AI technologies through three AI-specific governance disciplines that supplement the standard TPM governance framework. Model monitoring: establish continuous monitoring of AI model behavior in production for every AI platform in the portfolio, tracking model performance against defined metrics, detecting distribution shift between production inputs and training data, and alerting on behavioral changes that exceed defined tolerance thresholds. Model versioning governance: govern AI model updates with the same rigor applied to software version updates, requiring change documentation, performance validation, and governance approval before model updates are deployed to production. Explainability and transparency governance: require documented explainability approaches for AI platforms used in consequential decision-making — decisions that affect individuals’ access to services, financial terms, employment outcomes, or other significant interests — and assess whether the explainability capabilities of each AI platform are adequate for the regulatory requirements applicable to its use cases.

Benefit(s)

AI-specific governance disciplines integrated into the TPM framework produce an AI governance capability that is proportionate to the actual risks that AI technologies create and to the regulatory requirements that AI governance frameworks impose. Model monitoring prevents the silent degradation of AI system performance that ungoverned production deployment consistently produces. Model versioning governance prevents the uncontrolled behavioral changes that unmanaged model updates can introduce. Explainability governance ensures that consequential AI decisions can be explained and justified as regulatory and audit requirements increasingly demand.