Inventory Module - Production Quickstart Guide¶

Prerequisites¶

Required & Recommended

✅ AWS CLI configured with SSO profiles
✅ runbooks package installed (pip install runbooks or uv pip install runbooks)
✅ Access to AWS profiles with appropriate permissions

☑️ MANAGEMENT_PROFILE: AWS Organizations configured (for multi-account)
☑️ BILLING_PROFILE: Cost Explorer enabled (for financial intelligence)
☑️ CENTRALISED_OPS_PROFILE: Resource Explorer aggregator index (for cross-account discovery)

Quick Check:

## Verify runbooks installation. Expected: runbooks, version 1.1.22 (or higher)
runbooks --version

## Verify AWS CLI access. Expected: Account and user information
aws sts get-caller-identity --profile $MANAGEMENT_PROFILE

## Verify Task runner. Expected: Task version (any recent version)
task --version

2. CLI Commands & Classification (9 Categories)¶

2.1 Discovery Operations (6 tasks)¶

`discover-ec2` - EC2 Instance Discovery¶

task -t Taskfile.inventory.yaml discover-ec2

Discovers all EC2 instances across enabled regions.

`discover-rds` - RDS Database Discovery¶

task -t Taskfile.inventory.yaml discover-rds

Discovers all RDS instances and clusters.

`discover-s3` - S3 Bucket Discovery¶

task -t Taskfile.inventory.yaml discover-s3

Discovers all S3 buckets with metadata.

`discover-lambda` - Lambda Function Discovery¶

task -t Taskfile.inventory.yaml discover-lambda

Discovers all Lambda functions.

`discover-workspaces` - WorkSpaces Discovery¶

task -t Taskfile.inventory.yaml discover-workspaces

Discovers all WorkSpaces instances.

`list-resource-types` - Display Supported Resource Types¶

task -t Taskfile.inventory.yaml list-resource-types

Lists all 88 supported AWS resource types organized by category.

Example Output:

┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Alias              ┃ AWS Type               ┃ Description              ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ec2                │ EC2::Instance          │ Virtual servers          │
│ s3                 │ S3::Bucket             │ Object storage           │
│ rds                │ RDS::DBInstance        │ Relational databases     │
└────────────────────┴────────────────────────┴──────────────────────────┘

2.2 Organizations Operations (6 tasks)¶

`list-accounts` - List All AWS Accounts¶

task -t Taskfile.inventory.yaml list-accounts

Lists all accounts in AWS Organizations.

`draw-org` - Visualize Organization Hierarchy¶

task -t Taskfile.inventory.yaml draw-org

Rich tree visualization of organization structure.

`check-landing-zone` - Validate Landing Zone Configuration¶

task -t Taskfile.inventory.yaml check-landing-zone

Validates AWS Control Tower Landing Zone setup.

`check-control-tower` - Verify Control Tower Status¶

task -t Taskfile.inventory.yaml check-control-tower

Checks Control Tower deployment status and health.

`list-org-users` - List Organization IAM Users¶

task -t Taskfile.inventory.yaml list-org-users

Lists all IAM users across organization accounts.

`find-lz-versions` - Discover Landing Zone Versions¶

task -t Taskfile.inventory.yaml find-lz-versions

Identifies AWS Control Tower Landing Zone versions deployed.

2.3 Cost & Account Enrichment (2 tasks)¶

`enrich-accounts` - Add Organizations Metadata¶

task -t Taskfile.inventory.yaml enrich-accounts \
  INPUT=/tmp/ec2-discovered.csv \
  OUTPUT=/tmp/ec2-org.csv

Added Columns (7): - account_name: Friendly account name - account_email: Account email - wbs_code: Work breakdown structure - cost_group: Cost allocation group - technical_lead: Technical owner - account_owner: Business owner - organizational_unit: OU path

Example:

# Enrich with Organizations data
runbooks inventory enrich-accounts \
  --input /tmp/ec2-discovered.csv \
  --profile management-profile \
  --output /tmp/ec2-with-accounts.csv

`enrich-costs` - Add Cost Explorer Data¶

task -t Taskfile.inventory.yaml enrich-costs \
  INPUT=/tmp/ec2-org.csv \
  OUTPUT=/tmp/ec2-cost.csv \
  MONTHS=12

Added Columns (3): - monthly_cost: Current month cost (USD) - annual_cost_12mo: Trailing 12-month cost (USD) - cost_trend_3mo: 3-month trend (increasing/stable/decreasing)

Example:

# Add cost data with 12-month lookback
runbooks inventory enrich-costs \
  --input /tmp/ec2-with-accounts.csv \
  --profile billing-profile \
  --months 12 \
  --output /tmp/ec2-with-costs.csv

Advanced Options:

Option	Description	Example
`--months`	Trailing months	`12`
`--granularity`	MONTHLY or DAILY	`MONTHLY`
`--cost-metric`	AmortizedCost, UnblendedCost	`AmortizedCost`
`--skip-empty-costs`	Exclude $0 resources	-
`--cost-threshold`	Minimum monthly cost	`10.0`

2.4 Activity & Scoring Operations (3 tasks)¶

`enrich-activity` - Add Multi-API Activity Data¶

task -t Taskfile.inventory.yaml enrich-activity \
  INPUT=/tmp/ec2-cost.csv \
  RESOURCE_TYPE=ec2 \
  OUTPUT=/tmp/ec2-activity.csv

Added Columns (11):

CloudTrail (E3 Signal - 8 points): - last_activity_date: Most recent event - days_since_activity: Days since last event - activity_count_90d: Total events in window

CloudWatch (E2 Signal - 10 points): - p95_cpu_utilization: P95 CPU over period - p95_network_bytes: P95 network bytes

SSM (E4 Signal - 8 points - EC2 only): - ssm_ping_status: Online, Offline, ConnectionLost - ssm_last_ping_date: Last heartbeat timestamp - ssm_days_since_ping: Days since heartbeat

Compute Optimizer (E1 Signal - 60 points - EC2 only): - compute_optimizer_finding: Idle, Underprovisioned, Optimized - compute_optimizer_cpu_max: Max CPU utilization (14 days) - compute_optimizer_recommendation: Right-sizing recommendation

Example:

# Full enrichment with all signals
runbooks inventory enrich-activity \
  --input /tmp/ec2-with-costs.csv \
  --resource-type ec2 \
  --profile ops-profile \
  --output /tmp/ec2-fully-enriched.csv

# Fast enrichment (skip CloudTrail/SSM)
runbooks inventory enrich-activity \
  --input /tmp/ec2-with-costs.csv \
  --resource-type ec2 \
  --profile ops-profile \
  --skip-cloudtrail \
  --skip-ssm \
  --output /tmp/ec2-fast-activity.csv

`enrich-ec2` - EC2-Specific Enrichment (Alias)¶

task -t Taskfile.inventory.yaml enrich-ec2

Alias for enrich-activity --resource-type ec2.

`score-decommission` - Calculate Decommission Scores¶

task -t Taskfile.inventory.yaml score-decommission \
  INPUT=/tmp/ec2-activity.csv \
  RESOURCE_TYPE=ec2 \
  OUTPUT=/tmp/ec2-scored.csv

Added Columns (3): - decommission_score: 0-100 point score - decommission_tier: MUST (80-100) | SHOULD (50-79) | COULD (25-49) | KEEP (<25) - signal_breakdown: JSON showing triggered signals

EC2 Signal Scoring (E1-E7): - E1: Compute Optimizer idle (60 points) - BACKBONE - E2: CloudWatch CPU/Network (10 points) - E3: CloudTrail activity (8 points) - E4: SSM heartbeat (8 points) - E5: Service attachment (6 points) - E6: Storage I/O (5 points) - E7: Cost savings (3 points)

Example:

# High-confidence candidates (MUST tier: 80-100)
runbooks inventory score-decommission \
  --input /tmp/ec2-fully-enriched.csv \
  --resource-type ec2 \
  --score-threshold 80 \
  --output /tmp/ec2-must-decommission.csv

# High-cost idle (>$10/month, score ≥70)
runbooks inventory score-decommission \
  --input /tmp/ec2-fully-enriched.csv \
  --resource-type ec2 \
  --score-threshold 70 \
  --min-monthly-cost 10.0 \
  --output /tmp/ec2-high-value-targets.csv

2.5 Pipeline Operations (3 tasks)¶

`pipeline-5-layer` - Complete EC2 Pipeline¶

task -t Taskfile.inventory.yaml pipeline-5-layer

Executes complete 5-layer pipeline: Discovery → Organizations → Costs → Activity → Scoring.

Execution Time: ~10-15 minutes (137 EC2 instances typical)

Output Files: - data/outputs/ec2-discovered.csv (Layer 1) - data/outputs/ec2-org.csv (Layer 2) - data/outputs/ec2-cost.csv (Layer 3) - data/outputs/ec2-activity.csv (Layer 4) - data/outputs/ec2-scored.csv (Layer 5)

`pipeline-5-layer-workspaces` - Complete WorkSpaces Pipeline¶

task -t Taskfile.inventory.yaml pipeline-5-layer-workspaces

Executes 5-layer pipeline for WorkSpaces with W1-W6 signals.

`pipeline-summary` - Display Pipeline Results¶

task -t Taskfile.inventory.yaml pipeline-summary RESOURCE_TYPE=ec2

Shows rich summary of pipeline execution results.

2.6 Validation Operations (2 tasks)¶

`validate-mcp` - MCP Cross-Validation¶

task -t Taskfile.inventory.yaml validate-mcp RESOURCE_TYPE=ec2

Cross-validates cost calculations with MCP (Model Context Protocol) server.

Validation Process: 1. Extracts cost data from enriched CSV 2. Queries MCP cost-explorer server for same period 3. Calculates variance (% difference) 4. Reports accuracy metrics

Example Output:

┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Metric             ┃ CSV Value    ┃ MCP Value    ┃ Variance   ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Cost         │ $15,432.67   │ $15,401.23   │ 0.2%       │
│ Accuracy           │ -            │ -            │ 99.8%      │
└────────────────────┴──────────────┴──────────────┴────────────┘

✅ PASS: Accuracy 99.8% exceeds target (≥99.5%)

`validate-costs` - Cost Data Accuracy Validation¶

task -t Taskfile.inventory.yaml validate-costs \
  INPUT=/tmp/ec2-with-costs.csv

Validates cost data completeness and consistency.

Validation Checks: - Cost data completeness (% resources with cost data) - Cost range validation (outlier detection) - Trend consistency (monthly vs annual alignment) - Zero-cost resource detection

2.7 Workflow Templates (4 tasks)¶

`workflow-single-account` - Single-Account 4-Layer Pipeline¶

task -t Taskfile.inventory.yaml workflow-single-account

Layers: 1. Discovery (resource-explorer) 2. Costs (enrich-costs) 3. Activity (enrich-activity) 4. Scoring (score-decommission)

Execution Time: ~10-15 minutes

Customization:

task -t Taskfile.inventory.yaml workflow-single-account \
  CENTRALISED_OPS_PROFILE=my-ops-profile \
  OUTPUT_DIR=/tmp/inventory

`workflow-multi-account` - Multi-Account 5-Layer Pipeline¶

task -t Taskfile.inventory.yaml workflow-multi-account

Layers: 1. Discovery (--all-profiles) 2. Organizations (enrich-accounts) 3. Costs (enrich-costs) 4. Activity (enrich-activity) 5. Scoring (score-decommission)

Execution Time: ~20-30 minutes (67+ accounts, 500+ resources)

2.8 Utility Operations (3 tasks)¶

`clean-outputs` - Clean Output Directory¶

task -t Taskfile.inventory.yaml clean-outputs

Removes all generated output files from data/outputs/.

`show-profiles` - Display Configured Profiles¶

task -t Taskfile.inventory.yaml show-profiles

Shows AWS profiles configured in environment.

`list-outputs` - List Generated Files¶

task -t Taskfile.inventory.yaml list-outputs

Lists all output files in data/outputs/ directory.

3. Taskfile Automation Patterns¶

3.1 Common Workflow Patterns¶

Pattern 1: Single-Account Complete Pipeline¶

# Execute complete 4-layer pipeline
task -t Taskfile.inventory.yaml workflow-single-account

# Validate results
task -t Taskfile.inventory.yaml validate-mcp RESOURCE_TYPE=ec2

# Display summary
task -t Taskfile.inventory.yaml pipeline-summary RESOURCE_TYPE=ec2

Pattern 2: Multi-Account Landing Zone Discovery¶

# Execute 5-layer pipeline with Organizations
task -t Taskfile.inventory.yaml workflow-multi-account

# Validate cost accuracy
task -t Taskfile.inventory.yaml validate-costs \
  INPUT=data/outputs/ec2-cost.csv

# List all outputs
task -t Taskfile.inventory.yaml list-outputs

Pattern 3: Fast Discovery (Skip Enrichment)¶

# Discovery only
task -t Taskfile.inventory.yaml discover-ec2

# Add cost enrichment only (skip activity)
task -t Taskfile.inventory.yaml enrich-costs \
  INPUT=data/outputs/ec2-discovered.csv \
  OUTPUT=data/outputs/ec2-cost.csv

Pattern 4: High-Value Targets (Cost-Focused)¶

# Full enrichment
task -t Taskfile.inventory.yaml pipeline-5-layer

# Score with cost threshold
runbooks inventory score-decommission \
  --input data/outputs/ec2-activity.csv \
  --resource-type ec2 \
  --min-monthly-cost 50.0 \
  --score-threshold 70 \
  --output data/outputs/ec2-high-value.csv

3.2 Custom Taskfile Variables¶

Override default variables:

# Custom profiles
task -t Taskfile.inventory.yaml workflow-multi-account \
  MANAGEMENT_PROFILE=my-mgmt \
  BILLING_PROFILE=my-billing \
  CENTRALISED_OPS_PROFILE=my-ops

# Custom output directory
task -t Taskfile.inventory.yaml pipeline-5-layer \
  OUTPUT_DIR=/custom/path

# Custom lookback windows
task -t Taskfile.inventory.yaml enrich-activity \
  ACTIVITY_LOOKBACK_DAYS=30 \
  CLOUDWATCH_PERIOD=7

4. Integration Patterns¶

4.1 MCP Validation Integration¶

Purpose: Cross-validate inventory cost calculations.

Architecture:

┌─────────────────┐      ┌─────────────────┐      ┌─────────────────┐
│ Inventory       │      │ MCP Server      │      │ AWS Cost        │
│ Cost Enricher   │─────▶│ awslabs.cost-   │◀────▶│ Explorer API    │
│                 │      │ explorer        │      │                 │
└─────────────────┘      └─────────────────┘      └─────────────────┘
         │                        │
         ▼                        ▼
    CSV Output              JSON Response
  ($15,432.67)             ($15,401.23)
         │                        │
         └────────┬───────────────┘
                  │
                  ▼
          Variance Analysis
           (0.2% = 99.8% accuracy)

Configuration:

{
  "servers": {
    "awslabs-cost-explorer": {
      "command": "uv",
      "args": ["run", "mcp-server-awslabs-cost-explorer"],
      "env": {
        "AWS_PROFILE": "billing-profile",
        "AWS_REGION": "us-east-1"
      }
    }
  }
}

4.2 Jupyter Notebook Integration¶

Example Analysis:

import pandas as pd
import matplotlib.pyplot as plt

# Load enriched data
df = pd.read_csv('data/outputs/ec2-scored.csv')

# Filter high-confidence candidates
must_candidates = df[df['decommission_tier'] == 'MUST']

# Calculate potential savings
total_savings = must_candidates['monthly_cost'].sum()
print(f"Potential monthly savings: ${total_savings:,.2f}")

# Visualize score distribution
df['decommission_score'].hist(bins=20)
plt.xlabel('Decommission Score')
plt.ylabel('Frequency')
plt.title('EC2 Decommission Score Distribution')
plt.show()

4.3 Cost Explorer API Integration¶

Request Pattern:

response = ce_client.get_cost_and_usage(
    TimePeriod={
        'Start': '2024-11-01',
        'End': '2025-11-01'
    },
    Granularity='MONTHLY',
    Metrics=['AmortizedCost'],
    GroupBy=[
        {'Type': 'DIMENSION', 'Key': 'LINKED_ACCOUNT'}
    ]
)

Rate Limits: - 5 requests per second - 100 requests per day (free tier)

Best Practices: - Use MONTHLY granularity (fewer API calls) - Use AmortizedCost for enterprise RI/SP tracking - Cache responses for 24 hours (data latency)

5. Troubleshooting & FAQ¶

5.1 Common Issues¶

Issue: "Access Denied" errors¶

Solution:

# Verify IAM permissions
aws iam get-user --profile ops-profile
aws iam list-attached-user-policies --user-name your-username

# Required: ec2:Describe*, ce:GetCostAndUsage, cloudtrail:LookupEvents

Issue: "No Cost Data" in enrichment output¶

Solution:

# Verify Cost Explorer enabled
aws ce get-cost-and-usage \
  --time-period Start=2025-10-01,End=2025-11-01 \
  --granularity MONTHLY \
  --metrics AmortizedCost \
  --profile billing-profile

# Check resource age (must be >24 hours old)

Issue: CloudTrail API throttling¶

Solution:

# Option 1: Reduce lookback window
runbooks inventory enrich-activity \
  --activity-lookback-days 30 \
  --output /tmp/ec2-activity.csv

# Option 2: Skip CloudTrail
runbooks inventory enrich-activity \
  --skip-cloudtrail \
  --output /tmp/ec2-activity.csv

Issue: Timeout with large inventories¶

Solution:

# Option 1: Limit regions
runbooks inventory resource-explorer \
  --regions ap-southeast-2 us-east-1 \
  --output /tmp/ec2.csv

# Option 2: Filter by accounts
runbooks inventory resource-explorer \
  --accounts 123456789012,987654321098 \
  --output /tmp/ec2-filtered.csv

5.2 Frequently Asked Questions¶

Q: How long does a complete 5-layer pipeline take?

A: Typical execution times: - Single account (137 EC2): 10-15 minutes - Multi-account LZ (67 accounts, 500+ resources): 20-30 minutes

Q: Can I run inventory without Cost Explorer?

A: Yes. All enrichment layers optional:

# Discovery only
runbooks inventory resource-explorer --output /tmp/ec2.csv

# Activity only (skip costs)
runbooks inventory enrich-activity --output /tmp/ec2-activity.csv

Q: What's the difference between --profile and --all-profiles?

A: - --profile: Single-account operations - --all-profiles: Multi-account discovery via Resource Explorer aggregator

Q: How do I export in multiple formats?

A:

# Option 1: --all-outputs flag
runbooks inventory resource-explorer \
  --all-outputs \
  --output-dir /tmp/outputs

# Option 2: Specify format
runbooks inventory resource-explorer \
  --format pdf \
  --output /tmp/ec2.pdf

Q: Can I customize decommission signal weights?

A: Yes:

runbooks inventory score-decommission \
  --custom-weights '{"E1": 70, "E2": 5, "E7": 20}' \
  --output /tmp/ec2-custom-scores.csv

Q: What AWS regions are supported?

A: All enabled AWS regions. Use --all-regions for complete coverage:

runbooks inventory resource-explorer \
  --all-regions \
  --output /tmp/ec2-all-regions.csv

Q: How do I handle terminated/stopped resources?

A: Graceful degradation: - Terminated: Skipped in activity enrichment - Stopped: Included with state="stopped" - Cost data: Preserved for terminated (historical costs)

Q: Can I schedule inventory collection via cron?

A: Yes:

# Daily EC2 inventory at 2 AM
0 2 * * * cd /opt/cloudops && task -t Taskfile.inventory.yaml workflow-single-account >> /var/log/inventory.log 2>&1

# Weekly multi-account on Sundays at 3 AM
0 3 * * 0 cd /opt/cloudops && task -t Taskfile.inventory.yaml workflow-multi-account >> /var/log/inventory-lz.log 2>&1

5.3 Performance Optimization¶

Problem: Slow discovery across many regions

Solutions:

# 1. Limit regions
runbooks inventory resource-explorer \
  --regions ap-southeast-2 us-east-1

# 2. Skip pagination (preview mode)
runbooks inventory resource-explorer \
  --skip-pagination

# 3. Filter by tags
runbooks inventory resource-explorer \
  --tags Environment=prod

Problem: Slow activity enrichment

Solutions:

# Skip expensive APIs
runbooks inventory enrich-activity \
  --skip-cloudtrail \
  --skip-ssm

# Reduce lookback window
runbooks inventory enrich-activity \
  --activity-lookback-days 30 \
  --cloudwatch-period 7

Benchmark Results (137 EC2, 3 regions): | Operation | Time | Optimization | |-----------|------|-------------| | Discovery (all regions) | 90s | Limit to 2 regions: 30s (67% faster) | | Cost enrichment (DAILY) | 120s | Use MONTHLY: 60s (50% faster) | | Activity enrichment (full) | 180s | Skip CloudTrail+SSM: 90s (50% faster) | | Total (full) | 398s | Optimized: 188s (53% faster) |

Appendix A: AWS IAM Policy Templates¶

Discovery Profile Policy¶

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "ec2:Describe*",
      "rds:Describe*",
      "s3:List*",
      "resource-explorer-2:*",
      "cloudwatch:GetMetricStatistics",
      "compute-optimizer:GetEC2InstanceRecommendations"
    ],
    "Resource": "*"
  }]
}

Management Profile Policy¶

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "organizations:ListAccounts",
      "organizations:DescribeOrganization",
      "organizations:DescribeAccount"
    ],
    "Resource": "*"
  }]
}

Billing Profile Policy¶

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "ce:GetCostAndUsage",
      "ce:GetCostForecast"
    ],
    "Resource": "*"
  }]
}

Appendix B: Signal Reference¶

EC2 Decommission Signals (E1-E7)¶

E1: Compute Optimizer idle (60 points) - Backbone signal
E2: CloudWatch CPU/Network (10 points)
E3: CloudTrail activity (8 points)
E4: SSM heartbeat (8 points)
E5: Service attachment (6 points)
E6: Storage I/O (5 points)
E7: Cost savings (3 points)

WorkSpaces Decommission Signals (W1-W6)¶

W1: Connection recency (45 points) - Backbone signal
W2: CloudWatch usage (25 points)
W3: Billing vs usage (15 points)
W4: Cost Optimizer policy (10 points)
W5: Admin activity (5 points)
W6: User status (5 points)

Environment-Specific Commands¶

Development/Testing (Local Editable Install)¶

When: Active development, testing changes, or local debugging

# Use uv run for editable install (src/ directory changes reflected immediately)
uv run runbooks inventory resource-explorer --resource-type ec2:instance
uv run runbooks inventory enrich-costs --input data/outputs/ec2-discovered.csv

Why: - Changes to src/runbooks/ immediately available without reinstall - Ideal for rapid iteration and debugging - Full access to development dependencies

Production (SIT/PROD Environments)¶

When: Deployed environments, automation workflows, CI/CD pipelines

# Use runbooks directly (PyPI stable release)
runbooks inventory resource-explorer --resource-type ec2:instance
runbooks inventory enrich-costs --input data/outputs/ec2-discovered.csv

Why: - Stable PyPI release version - Consistent behavior across environments - No local source code dependencies - Faster startup (no uv overhead)

Version Verification:

# Check which command you're using
which runbooks  # Production: /usr/local/bin/runbooks or ~/.local/bin/runbooks
uv run runbooks --version  # Development: Shows local src/ version

Single Account Quickstart (5 minutes)¶

Use Case: Development, testing, or standalone AWS account

Step 1: Configure Profile (30 seconds)¶

# Set all profiles to your account
export CENTRALISED_OPS_PROFILE=my-account-profile
export MANAGEMENT_PROFILE=my-account-profile
export BILLING_PROFILE=my-account-profile

# Verify profile access
aws sts get-caller-identity --profile $CENTRALISED_OPS_PROFILE

Step 2: Discover Resources (2 minutes)¶

# Discover EC2 instances
task -t Taskfile.inventory.yaml discover-ec2

# Expected Output:
# 🔍 Discovering EC2 instances...
# ✅ EC2 discovery complete data/outputs/ec2-discovered.csv
#       137 data/outputs/ec2-discovered.csv

What happened: - Resource Explorer searched your AWS account - Found 137 EC2 instances (example) - Saved to CSV file for further enrichment

Step 3: Enrich with Costs (2 minutes)¶

# Add organization metadata
task -t Taskfile.inventory.yaml enrich-accounts

# Add 12-month cost data
task -t Taskfile.inventory.yaml enrich-costs

# Expected Output:
# 💰 Enriching with cost data (12-month history)...
# ✅ Cost enrichment complete data/outputs/ec2-cost.csv

What happened: - Added AWS Organizations account metadata - Enriched with 12-month cost trends from Cost Explorer - Calculated monthly cost averages and annual totals

Step 4: View Results (1 minute)¶

# Display pipeline summary
task -t Taskfile.inventory.yaml pipeline-summary

# Expected Output:
# Layer                          | Rows | Cols | Status
# ------------------------------------------------------------
# Layer 1 Discovery             |  137 |   10 | ✅
# Layer 2 Organizations         |  137 |   20 | ✅
# Layer 3 Costs                 |  137 |   23 | ✅
# ✅ Pipeline execution complete!

Business Value Delivered: - ✅ Complete EC2 inventory with account context - ✅ 12-month cost trends for financial planning - ✅ Export-ready CSV files for stakeholder reporting - ✅ Foundation for activity analysis and decommissioning

Quick Wins¶

View cost totals:

# Calculate total monthly costs
tail -n +2 data/outputs/ec2-cost.csv | cut -d',' -f15 | \
  awk '{sum+=$1} END {printf "Total Monthly Cost: $%.2f\n", sum}'

# Expected: Total Monthly Cost varies by environment

Export for Excel:

# CSV files ready for Excel/Google Sheets
open data/outputs/ec2-cost.csv

# Or copy to shared drive
cp data/outputs/ec2-cost.csv /path/to/shared/drive/

Multi-Account Quickstart (10 minutes)¶

Use Case: AWS Organizations with centralized operations

Step 1: Verify Multi-Account Setup (1 minute)¶

# Check Organizations access
aws organizations describe-organization --profile $MANAGEMENT_PROFILE

# Expected Output: Organization details with multiple accounts

# Check Resource Explorer aggregator
aws resource-explorer-2 list-indexes \
  --profile $CENTRALISED_OPS_PROFILE \
  --region ap-southeast-2

# Expected: At least one index with Type: AGGREGATOR

Step 2: List Organization Accounts (1 minute)¶

# Discover all AWS accounts in organization
task -t Taskfile.inventory.yaml list-accounts

# Expected Output:
# 📋 Listing AWS accounts...
# Account ID: 123456789012, Name: Production, Email: [email protected]
# Account ID: 987654321098, Name: Development, Email: [email protected]
# [... 65 more accounts ...]
# ✅ Account list complete

Business Value: - ✅ Complete account inventory for governance - ✅ Email contacts for accountability tracking - ✅ Foundation for multi-account resource discovery

Step 3: Run Complete 5-Layer Pipeline (5 minutes)¶

# Execute complete enrichment pipeline for EC2
task -t Taskfile.inventory.yaml pipeline-5-layer

# Pipeline Progress:
# 🔍 Layer 1: Resource Discovery... (2 min)
# 🏢 Layer 2: Organizations Enrichment... (30 sec)
# 💰 Layer 3: Cost Enrichment... (1 min)
# 📊 Layer 4: Activity Analysis... (1.5 min)
# 🎯 Layer 5: Decommission Scoring... (10 sec)

What Each Layer Delivers:

Layer	Time	Purpose	Output Columns	Business Value
1. Discovery	2 min	Find resources across 67 accounts	10	Complete resource inventory
2. Organizations	30 sec	Add account metadata (owner, WBS, cost center)	+10	Accountability and chargeback
3. Costs	1 min	12-month cost trends and forecasts	+3	Financial planning and budgeting
4. Activity	1.5 min	CloudTrail, CloudWatch, SSM, Compute Optimizer	+13	Idle resource detection
5. Scoring	10 sec	MUST/SHOULD/COULD/KEEP decommission tiers	+3	Prioritized cost reduction roadmap

Step 4: Analyze Results (2 minutes)¶

# View final pipeline summary
task -t Taskfile.inventory.yaml pipeline-summary

# Expected Output:
# Layer                          | Rows | Cols | Status
# ------------------------------------------------------------
# Layer 1 Discovery             |  137 |   10 | ✅
# Layer 2 Organizations         |  137 |   20 | ✅
# Layer 3 Costs                 |  137 |   23 | ✅
# Layer 4 Activity              |  137 |   36 | ✅
# Layer 5 Scored                |  137 |   39 | ✅
# ✅ Pipeline execution complete!

Decommission Tier Distribution:

# Analyze decommission recommendations
tail -n +2 data/outputs/ec2-scored.csv | cut -d',' -f30 | sort | uniq -c

# Expected Output:
#   11 MUST      (8% - high-confidence decommissioning)
#   19 SHOULD    (14% - strong candidates, requires review)
#   43 COULD     (31% - possible optimization, low priority)
#   64 KEEP      (47% - active resources, do not decommission)

Financial Impact:

# Calculate monthly savings potential
awk -F',' 'NR>1 && $30=="MUST" {sum+=$15; count++} END {
  printf "MUST tier: %d instances = $%.2f/month savings\n", count, sum
}' data/outputs/ec2-scored.csv

awk -F',' 'NR>1 && $30=="SHOULD" {sum+=$15; count++} END {
  printf "SHOULD tier: %d instances = $%.2f/month savings\n", count, sum
}' data/outputs/ec2-scored.csv

# Expected Output:
# MUST tier: 11 instances = $1,847.50/month savings
# SHOULD tier: 19 instances = $2,368.75/month savings
# Total potential: $4,216.25/month = $50,595/year

Common Use Cases¶

Use Case 1: Cost Optimization Initiative¶

Objective: Identify idle EC2 instances for decommissioning

Commands:

# Step 1: Complete 5-layer pipeline
task -t Taskfile.inventory.yaml pipeline-5-layer

# Step 2: Filter MUST tier instances (80-100 score)
awk -F',' 'NR==1 || $30=="MUST"' data/outputs/ec2-scored.csv > /tmp/ec2-decommission-must.csv

# Step 3: Review decommission signals
head -20 /tmp/ec2-decommission-must.csv | cut -d',' -f1,5,15,30,31,32

# Expected Columns:
# resource_arn, resource_id, monthly_cost, decommission_tier, decommission_score, signal_breakdown

Business Outcome: - ✅ Prioritized list of 11 instances (MUST tier) for immediate decommissioning - ✅ $1,847.50/month savings ($22,170/year) - ✅ Evidence-based justification (Compute Optimizer idle findings + low CPU/network + no CloudTrail activity)

Use Case 2: Compliance Audit¶

Objective: Validate resource ownership and tagging compliance

Commands:

# Step 1: Discover resources with account metadata
task -t Taskfile.inventory.yaml discover-ec2
task -t Taskfile.inventory.yaml enrich-accounts

# Step 2: Check account ownership population
tail -n +2 data/outputs/ec2-org.csv | cut -d',' -f8,9,13 | head -10

# Expected Columns:
# account_name, account_email, account_owner

Business Outcome: - ✅ Complete resource-to-account mapping - ✅ Identify untagged resources (ownership gaps) - ✅ Compliance reporting for governance audits

Use Case 3: Multi-Resource Discovery¶

Objective: Discover EC2, RDS, WorkSpaces, and Lambda across organization

Commands:

# Discover all resource types in parallel
task -t Taskfile.inventory.yaml discover-ec2 &
task -t Taskfile.inventory.yaml discover-rds &
task -t Taskfile.inventory.yaml discover-workspaces &
task -t Taskfile.inventory.yaml discover-lambda &
wait

# View all discoveries
ls -lh data/outputs/*-discovered.csv

# Expected Output:
# ec2-discovered.csv        (137 resources)
# rds-discovered.csv        (43 resources)
# workspaces-discovered.csv (122 resources)
# lambda-discovered.csv     (87 resources)

Business Outcome: - ✅ Complete infrastructure inventory across 4 service types - ✅ Foundation for multi-service cost optimization - ✅ 389 total resources discovered in < 5 minutes

Troubleshooting Quick Reference¶

Issue: No Resources Found¶

Symptoms:

✅ EC2 discovery complete data/outputs/ec2-discovered.csv
      1 data/outputs/ec2-discovered.csv  (header only, no data)

Solutions: 1. Verify profile has access to resources:

aws ec2 describe-instances --profile $CENTRALISED_OPS_PROFILE
# Should return instances, not empty

Check Resource Explorer configuration:

aws resource-explorer-2 list-indexes --profile $CENTRALISED_OPS_PROFILE
# Should show AGGREGATOR index

Verify region contains resources:

# Try different region
aws ec2 describe-instances --profile $CENTRALISED_OPS_PROFILE --region us-east-1

Issue: Cost Data Missing¶

Symptoms:

💰 Enriching with cost data (12-month history)...
WARNING: No cost data available for account 123456789012

Solutions: 1. Wait for Cost Explorer data availability (24 hours for new accounts) 2. Verify billing profile permissions:

aws ce get-cost-and-usage --profile $BILLING_PROFILE \
  --time-period Start=2025-10-01,End=2025-11-01 \
  --granularity MONTHLY \
  --metrics UnblendedCost

Use shorter time period for testing:

task -t Taskfile.inventory.yaml enrich-costs MONTHS=1

Issue: SSO Token Expired¶

Symptoms:

An error occurred (ExpiredToken) when calling the DescribeInstances operation

Solution:

# Re-authenticate with SSO
aws sso login --profile $CENTRALISED_OPS_PROFILE

# Verify token
aws sts get-caller-identity --profile $CENTRALISED_OPS_PROFILE

Issue: Permission Denied¶

Symptoms:

An error occurred (AccessDenied) when calling the Search operation

Solution:

# Check required permissions
# Layer 1: resource-explorer-2:Search
# Layer 2: organizations:ListAccounts
# Layer 3: ce:GetCostAndUsage
# Layer 4: cloudtrail:LookupEvents, cloudwatch:GetMetricStatistics

# Contact AWS administrator to add missing permissions

Next Steps¶

Expand to Other Resources¶

# Discover RDS databases
task -t Taskfile.inventory.yaml discover-rds
task -t Taskfile.inventory.yaml pipeline-5-layer RESOURCE_TYPE=rds

# Discover WorkSpaces
task -t Taskfile.inventory.yaml pipeline-5-layer-workspaces

# Discover S3 buckets
task -t Taskfile.inventory.yaml discover-s3

Automate with CI/CD¶

# .github/workflows/inventory-discovery.yml
name: Weekly Inventory Discovery

on:
  schedule:
    - cron: '0 0 * * 1'  # Every Monday at midnight

jobs:
  discover:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install runbooks
        run: pip install runbooks
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
      - name: Run inventory discovery
        run: task -t Taskfile.inventory.yaml pipeline-5-layer
      - name: Upload results
        uses: actions/upload-artifact@v3
        with:
          name: inventory-results
          path: data/outputs/ec2-scored.csv

Advanced Analysis¶

Jupyter Notebook Integration:

import pandas as pd

# Load scored results
df = pd.read_csv('data/outputs/ec2-scored.csv')

# Analyze by decommission tier
tier_distribution = df['decommission_tier'].value_counts()
print(tier_distribution)

# Calculate savings by tier
tier_savings = df.groupby('decommission_tier')['monthly_cost'].sum()
print(f"MUST tier savings: ${tier_savings['MUST']:.2f}/month")

Dashboard Visualization (PowerBI/Tableau):

# Export to Power BI compatible format
cp data/outputs/ec2-scored.csv ~/PowerBI/data/

# Or convert to Excel
pip install openpyxl
python << 'EOF'
import pandas as pd
df = pd.read_csv('data/outputs/ec2-scored.csv')
df.to_excel('ec2-scored.xlsx', index=False)
EOF

Help & Support¶

Documentation¶

Configuration Guide: /docs-runbooks/inventory/configuration-guide.md
Resource Explorer Guide: /docs-runbooks/inventory/resource-explorer.md
Inventory README: /src/runbooks/inventory/README.md

Commands Reference¶

# Show all available tasks
task -t Taskfile.inventory.yaml --list

# Display configured profiles
task -t Taskfile.inventory.yaml show-profiles

# View generated outputs
task -t Taskfile.inventory.yaml list-outputs

Common Tasks¶

Task	Command	Time
Single resource discovery	`task discover-ec2`	2 min
Complete 5-layer pipeline	`task pipeline-5-layer`	5 min
List organization accounts	`task list-accounts`	1 min
Generate org diagram	`task draw-org`	1 min
Clean output directory	`task clean-outputs`	5 sec

Quick Start Status: Production Ready ✅ Validated With: 67-account AWS Organizations (137 EC2, 122 WorkSpaces) Performance: <5 minutes complete 5-layer pipeline Accuracy: ≥99.5% (MCP validated) Last Updated: 2025-11-06

Inventory Module - Production Quickstart Guide¶

Prerequisites¶

2. CLI Commands & Classification (9 Categories)¶

2.1 Discovery Operations (6 tasks)¶

discover-ec2 - EC2 Instance Discovery¶

discover-rds - RDS Database Discovery¶

discover-s3 - S3 Bucket Discovery¶

discover-lambda - Lambda Function Discovery¶

discover-workspaces - WorkSpaces Discovery¶

list-resource-types - Display Supported Resource Types¶

2.2 Organizations Operations (6 tasks)¶

list-accounts - List All AWS Accounts¶

draw-org - Visualize Organization Hierarchy¶

check-landing-zone - Validate Landing Zone Configuration¶

check-control-tower - Verify Control Tower Status¶

list-org-users - List Organization IAM Users¶

find-lz-versions - Discover Landing Zone Versions¶

2.3 Cost & Account Enrichment (2 tasks)¶

enrich-accounts - Add Organizations Metadata¶

enrich-costs - Add Cost Explorer Data¶

2.4 Activity & Scoring Operations (3 tasks)¶

enrich-activity - Add Multi-API Activity Data¶

enrich-ec2 - EC2-Specific Enrichment (Alias)¶

score-decommission - Calculate Decommission Scores¶

2.5 Pipeline Operations (3 tasks)¶

pipeline-5-layer - Complete EC2 Pipeline¶

pipeline-5-layer-workspaces - Complete WorkSpaces Pipeline¶

pipeline-summary - Display Pipeline Results¶

2.6 Validation Operations (2 tasks)¶

validate-mcp - MCP Cross-Validation¶

validate-costs - Cost Data Accuracy Validation¶

2.7 Workflow Templates (4 tasks)¶

workflow-single-account - Single-Account 4-Layer Pipeline¶

workflow-multi-account - Multi-Account 5-Layer Pipeline¶

2.8 Utility Operations (3 tasks)¶

clean-outputs - Clean Output Directory¶

show-profiles - Display Configured Profiles¶

list-outputs - List Generated Files¶

3. Taskfile Automation Patterns¶

3.1 Common Workflow Patterns¶

Pattern 1: Single-Account Complete Pipeline¶

Pattern 2: Multi-Account Landing Zone Discovery¶

Pattern 3: Fast Discovery (Skip Enrichment)¶

Pattern 4: High-Value Targets (Cost-Focused)¶

3.2 Custom Taskfile Variables¶

4. Integration Patterns¶

4.1 MCP Validation Integration¶

4.2 Jupyter Notebook Integration¶

4.3 Cost Explorer API Integration¶

5. Troubleshooting & FAQ¶

5.1 Common Issues¶

Issue: "Access Denied" errors¶

Issue: "No Cost Data" in enrichment output¶

Issue: CloudTrail API throttling¶

Issue: Timeout with large inventories¶

5.2 Frequently Asked Questions¶

5.3 Performance Optimization¶

Appendix A: AWS IAM Policy Templates¶

Discovery Profile Policy¶

Management Profile Policy¶

Billing Profile Policy¶

Appendix B: Signal Reference¶

EC2 Decommission Signals (E1-E7)¶

WorkSpaces Decommission Signals (W1-W6)¶

Environment-Specific Commands¶

Development/Testing (Local Editable Install)¶

Production (SIT/PROD Environments)¶

Single Account Quickstart (5 minutes)¶

Step 1: Configure Profile (30 seconds)¶

Step 2: Discover Resources (2 minutes)¶

Step 3: Enrich with Costs (2 minutes)¶

Step 4: View Results (1 minute)¶

Quick Wins¶

Multi-Account Quickstart (10 minutes)¶

Step 1: Verify Multi-Account Setup (1 minute)¶

Step 2: List Organization Accounts (1 minute)¶

Step 3: Run Complete 5-Layer Pipeline (5 minutes)¶

Step 4: Analyze Results (2 minutes)¶

Common Use Cases¶

Use Case 1: Cost Optimization Initiative¶

`discover-ec2` - EC2 Instance Discovery¶

`discover-rds` - RDS Database Discovery¶

`discover-s3` - S3 Bucket Discovery¶

`discover-lambda` - Lambda Function Discovery¶

`discover-workspaces` - WorkSpaces Discovery¶

`list-resource-types` - Display Supported Resource Types¶

`list-accounts` - List All AWS Accounts¶

`draw-org` - Visualize Organization Hierarchy¶

`check-landing-zone` - Validate Landing Zone Configuration¶

`check-control-tower` - Verify Control Tower Status¶

`list-org-users` - List Organization IAM Users¶

`find-lz-versions` - Discover Landing Zone Versions¶

`enrich-accounts` - Add Organizations Metadata¶

`enrich-costs` - Add Cost Explorer Data¶

`enrich-activity` - Add Multi-API Activity Data¶

`enrich-ec2` - EC2-Specific Enrichment (Alias)¶

`score-decommission` - Calculate Decommission Scores¶

`pipeline-5-layer` - Complete EC2 Pipeline¶

`pipeline-5-layer-workspaces` - Complete WorkSpaces Pipeline¶

`pipeline-summary` - Display Pipeline Results¶

`validate-mcp` - MCP Cross-Validation¶

`validate-costs` - Cost Data Accuracy Validation¶

`workflow-single-account` - Single-Account 4-Layer Pipeline¶

`workflow-multi-account` - Multi-Account 5-Layer Pipeline¶

`clean-outputs` - Clean Output Directory¶

`show-profiles` - Display Configured Profiles¶

`list-outputs` - List Generated Files¶