Why This Matters

As organizations scale their testing efforts across multiple teams, projects, and environments, traditional CI/CD approaches struggle to keep pace. Test suites grow exponentially, pipeline execution times balloon, and teams face an impossible choice: run all tests and wait hours for feedback, or skip tests and risk production failures.

This lesson transforms you from a tactical test automation engineer into a strategic technology leader who can architect enterprise-scale solutions that leverage AI to intelligently optimize test execution, predict failures before they occur, and provide actionable insights across the entire software delivery lifecycle.

Real-World Problems You’ll Solve

Pipeline Bottlenecks at Scale: When your test suite reaches thousands of tests across microservices, running everything on every commit becomes untenable. You’ll learn to implement AI-powered test selection that identifies which tests actually need to run based on code changes, historical patterns, and risk analysis—reducing pipeline times by 60-80% without sacrificing quality.

Resource Waste and Cost Overruns: Cloud-based CI/CD environments charge by the minute for agent time. Inefficient test distribution and poor parallelization strategies can cost organizations tens of thousands of dollars monthly. You’ll master intelligent resource allocation strategies that optimize agent usage while maintaining optimal execution speed.

Siloed Testing Approaches: Most organizations treat functional, security, and performance testing as separate workflows with different tools and pipelines. This fragmentation leads to integration gaps, delayed feedback, and missed vulnerabilities. You’ll design unified AI testing frameworks that span all testing domains within a cohesive Azure DevOps architecture.

Lack of Executive Buy-In: Without clear metrics and demonstrable ROI, AI testing initiatives often fail to secure resources and organizational support. You’ll build comprehensive frameworks for measuring and communicating value in business terms that resonate with leadership.

When You’ll Use These Skills

Leading digital transformation initiatives where you need to modernize legacy testing approaches across multiple teams
Architecting CI/CD strategies for enterprise-scale organizations with complex deployment pipelines
Justifying tool investments by demonstrating clear ROI and business value to executives and stakeholders
Implementing governance frameworks that ensure consistency, compliance, and quality across distributed teams
Troubleshooting production incidents by leveraging AI analytics to identify patterns and predict future failures
Onboarding new teams to AI-powered testing practices with clear playbooks and change management strategies

What You’ll Accomplish

This advanced lesson takes you through a complete enterprise implementation journey, from technical integration to organizational transformation. You’ll build a production-ready framework that demonstrates leadership-level thinking and architectural sophistication.

Technical Mastery: Integration & Implementation

You’ll start by integrating AI-powered test selection directly into Azure DevOps pipelines, learning to configure intelligent agents that analyze code commits, historical test results, and risk factors to determine optimal test execution strategies. This goes beyond simple CI/CD setup—you’ll master advanced YAML pipeline configurations, service connections, and API integrations.

Next, you’ll configure sophisticated parallel execution strategies that intelligently distribute tests across multiple agents based on execution history, resource requirements, and dependency graphs. You’ll implement dynamic agent allocation, container-based test environments, and retry mechanisms that handle flaky tests gracefully.

The AI-driven analytics and reporting dashboards you create will provide actionable insights across test runs, including trend analysis, failure prediction, and quality metrics that feed directly into decision-making processes.

Strategic Expansion: Beyond Functional Testing

You’ll extend AI capabilities into security and performance domains, integrating tools like OWASP ZAP, Selenium with security plugins, and performance testing frameworks into your unified pipeline architecture. This cross-functional approach positions you as a holistic quality leader rather than a siloed automation specialist.

Leadership Development: Governance & Change Management

The final sections elevate you to strategic leadership by focusing on organizational transformation. You’ll design enterprise governance models that address compliance, standardization, and quality gates across multiple teams and projects.

Your ROI frameworks will include concrete metrics for measuring test execution efficiency, defect prevention value, resource cost savings, and time-to-market improvements—all presented in formats that resonate with business stakeholders.

Finally, you’ll develop comprehensive change management strategies that address cultural resistance, skill gaps, and adoption challenges. You’ll create communication plans, training frameworks, and success metrics that ensure your AI testing initiative succeeds organizationally, not just technically.

The Capstone: Enterprise-Ready Framework

By the lesson’s conclusion, you’ll have created a complete enterprise AI testing framework that includes:

Multi-stage Azure DevOps pipeline templates
Intelligent test selection and execution logic
Cross-domain testing integration (functional, security, performance)
Comprehensive analytics and reporting dashboards
Governance policies and compliance checkpoints
ROI measurement and business case documentation
Organizational adoption playbooks

This isn’t a proof-of-concept—it’s a production-ready architecture that demonstrates your ability to think strategically, lead technically, and drive organizational transformation at enterprise scale.

Core Content

Core Content: AI Testing in Azure DevOps Pipelines

Core Concepts Explained

Understanding AI Testing in CI/CD Context

AI testing in Azure DevOps Pipelines involves integrating automated test suites that leverage AI-powered tools to enhance test coverage, detect visual regressions, and improve test reliability. This advanced approach combines traditional test automation with intelligent test generation, self-healing locators, and predictive test analytics.

Key Components:

graph TD
    A[Azure DevOps Pipeline] --> B[Test Execution Stage]
    B --> C[AI-Powered Test Tools]
    C --> D[Playwright with Auto-Healing]
    C --> E[Visual AI Testing]
    C --> F[Test Analytics & Insights]
    D --> G[Test Results]
    E --> G
    F --> G
    G --> H[Pipeline Status]

Setting Up Azure DevOps Pipeline for AI Testing

Step 1: Create Azure Pipeline YAML Configuration

Create an azure-pipelines.yml file in your repository root:

trigger:
  branches:
    include:
      - main
      - develop

pool:
  vmImage: 'ubuntu-latest'

variables:
  PLAYWRIGHT_BROWSERS_PATH: 0

stages:
  - stage: Test
    displayName: 'AI-Enhanced Testing Stage'
    jobs:
      - job: AITests
        displayName: 'Run AI-Powered Tests'
        steps:
          - task: NodeTool@0
            inputs:
              versionSpec: '18.x'
            displayName: 'Install Node.js'

          - script: |
              npm ci
            displayName: 'Install Dependencies'

          - script: |
              npx playwright install --with-deps
            displayName: 'Install Playwright Browsers'

          - script: |
              npm run test:ai
            displayName: 'Execute AI Tests'
            env:
              APPLITOOLS_API_KEY: $(APPLITOOLS_API_KEY)
              TEST_ENV: 'ci'

          - task: PublishTestResults@2
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: '**/test-results/*.xml'
              mergeTestResults: true
              failTaskOnFailedTests: true
            displayName: 'Publish Test Results'
            condition: succeededOrFailed()

          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: 'test-results'
              artifact: 'test-artifacts'
            displayName: 'Publish Test Artifacts'
            condition: succeededOrFailed()

Step 2: Configure AI-Powered Test Framework

Create a Playwright configuration with AI-enhanced capabilities:

// playwright.config.js
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 2 : undefined,
  
  reporter: [
    ['html'],
    ['junit', { outputFile: 'test-results/junit.xml' }],
    ['json', { outputFile: 'test-results/results.json' }]
  ],

  use: {
    baseURL: 'https://practiceautomatedtesting.com',
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
    
    // AI-enhanced configuration
    actionTimeout: 10000,
    navigationTimeout: 30000,
  },

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'mobile-chrome',
      use: { ...devices['Pixel 5'] },
    },
  ],
});

Implementing AI-Powered Test Scenarios

Self-Healing Locators with AI

// tests/helpers/ai-locator.js
/**
 * AI-enhanced locator that falls back to alternative strategies
 */
export class AILocator {
  constructor(page) {
    this.page = page;
  }

  /**
   * Intelligent element location with multiple fallback strategies
   */
  async findElement(primaryLocator, context = {}) {
    const strategies = [
      // Primary locator
      () => this.page.locator(primaryLocator),
      
      // Text-based fallback
      () => context.text ? this.page.getByText(context.text) : null,
      
      // Role-based fallback
      () => context.role ? this.page.getByRole(context.role, { name: context.name }) : null,
      
      // Visual position fallback
      () => context.nearText ? this.page.locator(`text=${context.nearText}`).locator('..').locator(primaryLocator.split(' ').pop()) : null,
    ];

    for (const strategy of strategies) {
      try {
        const element = strategy();
        if (element) {
          await element.waitFor({ timeout: 5000 });
          return element;
        }
      } catch (error) {
        continue; // Try next strategy
      }
    }

    throw new Error(`Could not locate element: ${primaryLocator}`);
  }
}

Visual AI Testing Integration

// tests/visual-ai.spec.js
import { test, expect } from '@playwright/test';
import { Eyes, Target } from '@applitools/eyes-playwright';

test.describe('Visual AI Testing', () => {
  let eyes;

  test.beforeEach(async ({ page }) => {
    // Initialize Applitools Eyes
    eyes = new Eyes();
    eyes.setApiKey(process.env.APPLITOOLS_API_KEY);
    
    await eyes.open(page, 'Practice Testing App', test.info().title, {
      width: 1024,
      height: 768
    });
  });

  test.afterEach(async () => {
    await eyes.close();
  });

  test('homepage visual regression with AI', async ({ page }) => {
    await page.goto('https://practiceautomatedtesting.com');
    
    // AI-powered visual checkpoint
    await eyes.check('Homepage Full Page', Target.window().fully());
    
    // Check specific region with AI
    await eyes.check('Navigation Menu', 
      Target.region(page.locator('nav')).fully()
    );
  });

  test('product page responsive visual test', async ({ page }) => {
    await page.goto('https://practiceautomatedtesting.com/product/1');
    
    // AI compares across different viewports
    await eyes.check('Product Page - Desktop', Target.window().fully());
    
    await page.setViewportSize({ width: 375, height: 667 });
    await eyes.check('Product Page - Mobile', Target.window().fully());
  });
});

Smart Test Data Generation with AI Patterns

// tests/helpers/test-data-generator.js
/**
 * AI-inspired test data generation
 */
export class SmartTestDataGenerator {
  
  /**
   * Generate realistic user data with patterns
   */
  static generateUserData(scenario = 'default') {
    const patterns = {
      default: {
        email: `testuser_${Date.now()}@test.com`,
        password: 'Test@123456',
        name: 'Test User'
      },
      special_chars: {
        email: `test+special_${Date.now()}@test.com`,
        password: 'P@ssw0rd!#$',
        name: "O'Brien-Smith"
      },
      boundary: {
        email: `${'a'.repeat(50)}@test.com`,
        password: 'a'.repeat(100),
        name: 'A'.repeat(255)
      }
    };

    return patterns[scenario] || patterns.default;
  }

  /**
   * Generate edge case scenarios
   */
  static generateEdgeCases(fieldType) {
    const edgeCases = {
      email: [
        'test@test.com',
        'test+tag@test.co.uk',
        'test.name@subdomain.test.com',
        'invalid-email',
        '',
        ' '
      ],
      password: [
        'ValidPass123!',
        'short',
        'nouppercase123!',
        'NOLOWERCASE123!',
        'NoSpecialChar123',
        ''
      ]
    };

    return edgeCases[fieldType] || [];
  }
}

Complete AI-Enhanced Test Example

// tests/ai-enhanced-checkout.spec.js
import { test, expect } from '@playwright/test';
import { AILocator } from './helpers/ai-locator';
import { SmartTestDataGenerator } from './helpers/test-data-generator';

test.describe('AI-Enhanced Checkout Flow', () => {
  let aiLocator;

  test.beforeEach(async ({ page }) => {
    aiLocator = new AILocator(page);
    await page.goto('https://practiceautomatedtesting.com');
  });

  test('complete checkout with self-healing locators', async ({ page }) => {
    // AI-powered product selection
    const productCard = await aiLocator.findElement(
      '.product-card',
      { 
        text: 'Combination Pliers',
        role: 'article'
      }
    );
    await productCard.click();

    // Add to cart with fallback strategies
    const addToCartBtn = await aiLocator.findElement(
      '[data-test="add-to-cart"]',
      {
        role: 'button',
        text: 'Add to Cart',
        nearText: 'Combination Pliers'
      }
    );
    await addToCartBtn.click();

    // Verify cart badge updated
    await expect(page.locator('.cart-badge')).toContainText('1');

    // Navigate to checkout
    const checkoutBtn = await aiLocator.findElement(
      '#checkout-button',
      {
        role: 'link',
        text: 'Checkout'
      }
    );
    await checkoutBtn.click();

    // Fill checkout form with generated data
    const userData = SmartTestDataGenerator.generateUserData('default');
    
    await page.fill('[data-test="email"]', userData.email);
    await page.fill('[data-test="name"]', userData.name);
    await page.fill('[data-test="address"]', '123 Test Street');
    
    // Submit with AI-assisted validation
    await page.click('[data-test="submit-order"]');
    
    // Wait for confirmation with intelligent timeout
    await expect(page.locator('.order-confirmation')).toBeVisible({ timeout: 15000 });
  });

  test('test multiple edge cases with AI data generation', async ({ page }) => {
    await page.goto('https://practiceautomatedtesting.com/login');

    const emailEdgeCases = SmartTestDataGenerator.generateEdgeCases('email');
    
    for (const email of emailEdgeCases) {
      await page.fill('[data-test="email"]', email);
      await page.fill('[data-test="password"]', 'TestPassword123!');
      await page.click('[data-test="login-button"]');
      
      if (email.includes('@') && email.length > 5) {
        // Valid email format
        await expect(page.locator('.error-message')).not.toBeVisible();
      } else {
        // Invalid email format
        await expect(page.locator('.error-message')).toBeVisible();
      }
      
      await page.reload();
    }
  });
});

Setting Up Pipeline Variables and Secrets

# Add secrets to Azure DevOps Pipeline
az pipelines variable create --name APPLITOOLS_API_KEY \
  --value "your-api-key-here" \
  --secret true \
  --org https://dev.azure.com/your-org \
  --project your-project \
  --pipeline-name ai-testing-pipeline

Monitoring and Analytics

// tests/analytics/test-reporter.js
/**
 * Custom reporter for AI test insights
 */
export class AITestReporter {
  
  constructor() {
    this.testResults = [];
    this.aiInsights = {
      healedLocators: 0,
      visualDifferences: 0,
      performanceIssues: []
    };
  }

  onTestEnd(test, result) {
    this.testResults.push({
      title: test.title,
      status: result.status,
      duration: result.duration,
      retries: result.retry,
      errors: result.errors
    });

    // Track self-healing instances
    if (result.attachments.some(a => a.name.includes('healed-locator'))) {
      this.aiInsights.healedLocators++;
    }
  }

  async onEnd() {
    const report = {
      totalTests: this.testResults.length,
      passed: this.testResults.filter(t => t.status === 'passed').length,
      failed: this.testResults.filter(t => t.status === 'failed').length,
      aiInsights: this.aiInsights,
      timestamp: new Date().toISOString()
    };

    console.log('AI Test Report:', JSON.stringify(report, null, 2));
    
    // Write to file for pipeline artifact
    await require('fs').promises.writeFile(
      'test-results/ai-insights.json',
      JSON.stringify(report, null, 2)
    );
  }
}

Common Mistakes Section

What to Avoid

Over-reliance on AI without validation

// ❌ Bad: Blindly trusting AI-healed locators
const element = await aiLocator.findElement('.btn');
await element.click();

// ✅ Good: Validate before action
const element = await aiLocator.findElement('.btn');
await expect(element).toBeVisible();
await expect(element).toBeEnabled();
await element.click();

Not handling API key security properly

# ❌ Bad: Hardcoded API keys
env:
  APPLITOOLS_API_KEY: "abc123key"

# ✅ Good: Using pipeline variables
env:
  APPLITOOLS_API_KEY: $(APPLITOOLS_API_KEY)

Ignoring baseline management in visual AI testing

// ❌ Bad: No baseline strategy
await eyes.check('Page', Target.window());

// ✅ Good: Named baselines with versioning
await eyes.check('Homepage - v2.1', Target.window().fully(), {
  baseline: 'homepage-baseline-v2'
});

How to Debug Issues

Pipeline Failures:

# Check Azure DevOps logs
az pipelines runs show --id <run-id> --org https://dev.azure.com/your-org

# Download artifacts locally
az pipelines runs artifact download --run-id <run-id> --artifact-name test-artifacts

Visual AI Test Failures:

// Enable detailed logging
eyes.setLogHandler(new ConsoleLogHandler(true));

// Save debug screenshots
await page.screenshot({ path: `debug-${Date.now()}.png`, fullPage: true });

Self-Healing Locator Issues:

Hands-On Practice

Hands-On Exercise

🎯 Exercise: Build an AI Model Testing Pipeline in Azure DevOps

Objective

Create a complete Azure DevOps pipeline that trains, validates, and deploys a machine learning model with comprehensive testing gates at each stage.

Scenario

You’re deploying a customer churn prediction model. The pipeline must validate model performance, test API endpoints, and ensure model fairness before production deployment.

Prerequisites

Azure DevOps account with pipeline access
Azure ML workspace
Python 3.8+ environment
Basic ML model (provided in starter code)

📋 Task Requirements

Build an Azure DevOps pipeline that:

Trains a classification model
Validates model metrics (accuracy > 0.85, F1-score > 0.80)
Tests for model bias across demographic groups
Deploys model to staging endpoint
Runs integration tests against the API
Requires manual approval before production
Monitors deployment health

🔨 Step-by-Step Instructions

Step 1: Set Up Repository Structure

Create the following structure in your Azure DevOps repository:

project-root/
├── azure-pipelines.yml
├── src/
│   ├── train_model.py
│   ├── test_model.py
│   └── test_api.py
├── tests/
│   ├── test_bias.py
│   └── test_performance.py
├── deployment/
│   └── deploy_config.yml
└── requirements.txt

Step 2: Create Model Training Script

File: src/train_model.py

import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, f1_score, classification_report
import joblib
import json
import sys

def train_model():
    # Load data
    data = pd.read_csv('data/customer_data.csv')
    X = data.drop(['churn', 'customer_id'], axis=1)
    y = data['churn']
    
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.2, random_state=42, stratify=y
    )
    
    # Train model
    model = RandomForestClassifier(n_estimators=100, random_state=42)
    model.fit(X_train, y_train)
    
    # Evaluate
    y_pred = model.predict(X_test)
    metrics = {
        'accuracy': accuracy_score(y_test, y_pred),
        'f1_score': f1_score(y_test, y_pred),
        'test_size': len(y_test)
    }
    
    # Save model and metrics
    joblib.dump(model, 'outputs/model.pkl')
    with open('outputs/metrics.json', 'w') as f:
        json.dump(metrics, f)
    
    print(f"Model trained. Accuracy: {metrics['accuracy']:.3f}")
    
    return metrics

if __name__ == "__main__":
    metrics = train_model()
    # Exit with error if metrics don't meet threshold
    if metrics['accuracy'] < 0.85 or metrics['f1_score'] < 0.80:
        print("Model performance below threshold!")
        sys.exit(1)

Step 3: Create Bias Testing Script

File: tests/test_bias.py

import pandas as pd
import joblib
from sklearn.metrics import accuracy_score
import sys

def test_model_fairness():
    # Load model and test data
    model = joblib.load('outputs/model.pkl')
    data = pd.read_csv('data/customer_data.csv')
    
    # Test performance across demographic groups
    results = {}
    for group in data['demographic_group'].unique():
        group_data = data[data['demographic_group'] == group]
        X = group_data.drop(['churn', 'customer_id', 'demographic_group'], axis=1)
        y = group_data['churn']
        
        predictions = model.predict(X)
        accuracy = accuracy_score(y, predictions)
        results[group] = accuracy
    
    # Check fairness: max difference between groups < 0.05
    max_diff = max(results.values()) - min(results.values())
    
    print("Fairness Results:")
    for group, acc in results.items():
        print(f"  {group}: {acc:.3f}")
    print(f"Max difference: {max_diff:.3f}")
    
    if max_diff > 0.05:
        print("FAILED: Model shows bias across demographic groups")
        sys.exit(1)
    else:
        print("PASSED: Model fairness validated")
        
if __name__ == "__main__":
    test_model_fairness()

Step 4: Create API Integration Tests

File: tests/test_api.py

import requests
import json
import time
import sys

def test_api_endpoint(endpoint_url, api_key):
    """Test deployed model endpoint"""
    
    # Test cases
    test_data = [
        {"tenure": 12, "monthly_charges": 50.0, "total_charges": 600.0},
        {"tenure": 24, "monthly_charges": 80.0, "total_charges": 1920.0}
    ]
    
    headers = {
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {api_key}'
    }
    
    passed = 0
    failed = 0
    
    for i, payload in enumerate(test_data):
        try:
            response = requests.post(
                endpoint_url,
                headers=headers,
                json=payload,
                timeout=10
            )
            
            if response.status_code == 200:
                result = response.json()
                assert 'prediction' in result
                assert 'probability' in result
                assert 0 <= result['probability'] <= 1
                print(f"Test case {i+1}: PASSED")
                passed += 1
            else:
                print(f"Test case {i+1}: FAILED - Status {response.status_code}")
                failed += 1
                
        except Exception as e:
            print(f"Test case {i+1}: FAILED - {str(e)}")
            failed += 1
    
    print(f"\nResults: {passed} passed, {failed} failed")
    
    if failed > 0:
        sys.exit(1)

if __name__ == "__main__":
    endpoint = sys.argv[1]
    api_key = sys.argv[2]
    test_api_endpoint(endpoint, api_key)

Step 5: Create Azure Pipeline

File: azure-pipelines.yml

trigger:
  branches:
    include:
    - main
  paths:
    include:
    - src/*
    - tests/*

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: ml-credentials
  - name: modelName
    value: 'churn-prediction-model'

stages:
- stage: Build_and_Test
  displayName: 'Build and Test Model'
  jobs:
  - job: TrainModel
    displayName: 'Train ML Model'
    steps:
    - task: UsePythonVersion@0
      inputs:
        versionSpec: '3.8'
        
    - script: |
        pip install -r requirements.txt
      displayName: 'Install dependencies'
      
    - script: |
        python src/train_model.py
      displayName: 'Train model'
      
    - task: PublishPipelineArtifact@1
      inputs:
        targetPath: 'outputs'
        artifact: 'model-artifacts'
        
  - job: ValidateModel
    displayName: 'Validate Model Quality'
    dependsOn: TrainModel
    steps:
    - task: DownloadPipelineArtifact@2
      inputs:
        artifact: 'model-artifacts'
        path: 'outputs'
        
    - script: |
        python tests/test_bias.py
      displayName: 'Test model fairness'
      
    - script: |
        python tests/test_performance.py
      displayName: 'Validate performance metrics'

- stage: Deploy_Staging
  displayName: 'Deploy to Staging'
  dependsOn: Build_and_Test
  condition: succeeded()
  jobs:
  - deployment: DeployStaging
    displayName: 'Deploy to Staging Environment'
    environment: 'ml-staging'
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureCLI@2
            displayName: 'Deploy model to Azure ML'
            inputs:
              azureSubscription: $(azureServiceConnection)
              scriptType: 'bash'
              scriptLocation: 'inlineScript'
              inlineScript: |
                az ml model deploy \
                  --name $(modelName)-staging \
                  --model outputs/model.pkl \
                  --inference-config deployment/inference_config.yml \
                  --deployment-config deployment/deploy_config.yml \
                  --workspace-name $(mlWorkspace) \
                  --resource-group $(resourceGroup)
                  
          - script: |
              sleep 60  # Wait for endpoint warmup
              python tests/test_api.py $(stagingEndpoint) $(apiKey)
            displayName: 'Run integration tests'

- stage: Deploy_Production
  displayName: 'Deploy to Production'
  dependsOn: Deploy_Staging
  condition: succeeded()
  jobs:
  - deployment: DeployProduction
    displayName: 'Deploy to Production'
    environment: 'ml-production'
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureCLI@2
            displayName: 'Deploy to production'
            inputs:
              azureSubscription: $(azureServiceConnection)
              scriptType: 'bash'
              scriptLocation: 'inlineScript'
              inlineScript: |
                az ml model deploy \
                  --name $(modelName)-production \
                  --model outputs/model.pkl \
                  --inference-config deployment/inference_config.yml \
                  --deployment-config deployment/deploy_config.yml \
                  --workspace-name $(mlWorkspace) \
                  --resource-group $(resourceGroup)
                  
          - task: PublishTestResults@2
            displayName: 'Publish test results'
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: '**/test-results.xml'

- stage: Monitor
  displayName: 'Post-Deployment Monitoring'
  dependsOn: Deploy_Production
  jobs:
  - job: HealthCheck
    displayName: 'Monitor Model Health'
    steps:
    - script: |
        python tests/monitor_endpoint.py $(productionEndpoint) $(apiKey)
      displayName: 'Run health checks'

Step 6: Configure Variable Groups

In Azure DevOps:

Navigate to Pipelines → Library
Create variable group ml-credentials
Add variables:
- azureServiceConnection
- mlWorkspace
- resourceGroup
- apiKey (mark as secret)
- stagingEndpoint
- productionEndpoint

Step 7: Set Up Environments with Approvals

Go to Pipelines → Environments
Create ml-staging (automatic deployment)
Create ml-production with manual approval gate
Add approvers for production environment

✅ Expected Outcomes

After completing this exercise, you should have:

✅ A working multi-