Collection Quality Scoring

At the end of collection, the Data Collector computes a quality score and writes it to the collection_quality block of the output JSON. This block feeds the admin notification system, drives the quality summary in the client report, and gives the Assessment Engine context about data completeness before scoring begins.

Quality Score Calculation

quality_score = (assessable_items / (total_items - not_applicable_items)) × 100

assessable_items  = compliant + non_compliant + b2_data_present + b3_answered
total_items       = all items across selected checklists
not_applicable    = items where resource type not deployed in tenant

Quality Thresholds

Score	Status	Meaning
90–100%	Excellent	Full assessment. All critical data sources available. Report findings are highly reliable.
75–89%	Good	Strong assessment. Minor data gaps. Report findings are reliable with noted exceptions.
60–74%	Partial	Moderate data gaps. Likely missing permissions for one or more domains. Report includes quality advisory.
Below 60%	Degraded	Significant data gaps. Admin notification triggered. Report includes prominent quality warning. Client advised to grant additional permissions and re-run.

collection_quality Block Schema

{
  "quality_score":         87,
  "quality_status":        "Good",
  "total_items":           255,
  "assessable_items":      221,
  "not_applicable_items":    8,
  "data_unavailable_items": 26,
  "error_count":             4,
  "critical_gaps": [
    {
      "domain":         "Defender for Cloud",
      "reason":         "Security Reader role not granted",
      "items_affected": 14
    }
  ],
  "sentinel_deployed":     false,
  "collection_duration_s": 48
}

Admin Notification

Every completed job triggers an admin notification. Elevated priority is triggered when:

• Quality score below 60%
• Collection duration exceeded 90 seconds
• More than 10 errors in collection_errors
• Phase 1 failure (critical alert — job marked failed)

See Admin Notifications for the full notification payload and response runbook.