Skip to content

Configuration Best Practices Guide

🎯 Profile management strategies for single-account and multi-account AWS environments
  • Single-account
  • Multi-account LZ: Management, Billing, Centralised-Ops 
---
title: Single-Account vs Multi-Account Profile - Enterprise AWS Architecture
config:
  theme: base
  themeVariables:
    background: '#f2f6fa'
    primaryColor: '#2563eb'
    edgeLabelBackground: '#ffffff'
    fontFamily: 'Segoe UI, Arial, sans-serif'
    fontSize: 17px
---
flowchart TB
    %% TOP LAYER: CLI TRIGGER
    CLI["Runbooks CLI<br/><b>runbooks inventory</b><br/>--all-profiles --outputs"]:::cli

    %% MIDDLE LAYER: AWS ACCOUNTS/PROFILES
    subgraph  AWS_Accounts["Multi-Account LZ"]
        MgmtAcct["🏢 Management Account & SSO<br/>✓ Organization APIs"]:::mgmt
        BillingAcct["💰 Billing Account<br/>(Cost Explorer API)<br/>✓ Consolidated Billing"]:::billing
        OpsAcct["⚙️ Centralised-Ops<br/>Cross-account Ops<br/>✓ Resource Explorer"]:::ops
    end

    subgraph "Single Account Testing"
        TestAcct["🧪 Test SRE Account<br/>Runbooks + AWS-APIs<br/>✓ AWS APIs"]:::test
    end

    %% BOTTOM LAYER: API/RESULTS FOR EACH ACCOUNT
    MgmtAcct -->|Organizations API| OrgAPI["<b>AWS Organizations</b><br>✓ Multi-accounts<br>✓ OU hierarchy<br>✓ 30-min cache"]:::api
    BillingAcct -->|Cost Explorer API| CostAPI["<b>AWS Cost Explorer</b><br>✓ 3-way validated<br>✓ 12mo History<br>✓ Account-level costs"]:::api
    OpsAcct -->|Resource Explorer| ResAPI["<b>AWS Resource Explorer</b><br>✓ 88 resource types<br>✓ EC2, RDS, S3, Lambda<br/>✓ Cross-region discovery"]:::api
    TestAcct -->|Direct APIs| TestAPI["<b>Autonomous Validation</b><br>✓ Single Account Testing<br>✓ No Org dependency<br>✓ Real resource coverage"]:::api

    %% CONNECT CLI TO ALL ACCOUNTS WITH LABELS
    CLI -- "MANAGEMENT_PROFILE" --> MgmtAcct
    CLI -- "BILLING_PROFILE" --> BillingAcct
    CLI -- "CENTRALISED_OPS_PROFILE" --> OpsAcct
    CLI -- "AWS_PROFILE" --> TestAcct

    %% CLASSES FOR HIGH CONTRAST, PROFESSIONAL COLORS
    classDef cli     fill:gray,stroke:#2d4464,color:green,stroke-width:1px;
    classDef mgmt    fill:#e53935,stroke:#a5271a,color:#fff,stroke-width:1px;
    classDef billing fill:#f9a825,stroke:#bf760d,color:#222,stroke-width:1px;
    classDef ops     fill:#1976d2,stroke:#0d3563,color:#fff,stroke-width:1px;
    classDef test    fill:#607d8b,stroke:#455a64,color:#fff,stroke-width:1px;
    classDef api     fill:#fff,stroke:#678abd,color:#212,font-size:15px,stroke-width:2px;

    %% link style for consistency and visibility
    linkStyle default stroke:#444,stroke-width:2px;

1. Single Account Configuration

Use Case: Development, testing, or standalone AWS account scenarios

When to Use Single Account Mode

✅ Recommended for: - Development and testing environments - Proof-of-concept validation - Single AWS account organizations - Learning and experimentation

❌ Not recommended for: - Multi-account AWS Organizations (use multi-account mode instead) - Cross-account resource discovery - Enterprise Landing Zone deployments

Example: Single Account Discovery

# Step 1: Configure single profile
export AWS_PROFILE="my-profile-ReadOnly"

# Step 2: Verify profile access
aws sts get-caller-identity --profile $AWS_PROFILE

# Step 3: Run 5-layer pipeline
task -t Taskfile.inventory.yaml pipeline-5-layer

# Expected Output:
# Layer 1 Discovery           |   137 |   10 | ✅
# Layer 2 Organizations        |   137 |   20 | ✅
# Layer 3 Costs                |   137 |   23 | ✅
# Layer 4 Activity             |   137 |   36 | ✅
# Layer 5 Scored               |   137 |   39 | ✅

Validation: 

# Verify resources discovered in single account only
tail -n +2 data/outputs/ec2-scored.csv | cut -d',' -f2 | sort -u | wc -l
# Expected: 1 (single account ID)

2. Multi-Account Landing Zone Configuration

Use Case: AWS Organizations with centralized operations and billing

Configuration Pattern

# Uses Taskfile defaults (production-tested with 67 accounts)
task -t Taskfile.inventory.yaml discover-ec2
task -t Taskfile.inventory.yaml list-accounts
task -t Taskfile.inventory.yaml enrich-costs

Default Profile Architecture

The Taskfile.inventory.yaml provides production-validated defaults:

Override when needed: 

# Override one or more profiles for your environment
export CENTRALISED_OPS_PROFILE="your-ops-profile"
export MANAGEMENT_PROFILE="your-management-profile"
export BILLING_PROFILE="your-billing-profile"

When to Use Multi-Account Mode

✅ Recommended for: - AWS Organizations with multiple accounts - Enterprise Landing Zone deployments - AWS Control Tower managed environments - Centralized operations teams

✅ Prerequisites: - AWS Organizations enabled - Resource Explorer aggregator index configured - Management account access for Organizations API - Billing account access for Cost Explorer

Example: Multi-Account Discovery

# Step 1: Verify profile access
aws organizations describe-organization --profile $MANAGEMENT_PROFILE
# Expected: Organization details with multiple accounts

# Step 2: Check Resource Explorer aggregator
aws resource-explorer-2 list-indexes \
  --profile $CENTRALISED_OPS_PROFILE \
  --region ap-southeast-2
# Expected: At least one index with Type: AGGREGATOR

# Step 3: Run complete pipeline
task -t Taskfile.inventory.yaml pipeline-5-layer

# Expected: Discovery across ALL organization accounts

Validation: 

# Count unique accounts discovered
tail -n +2 data/outputs/ec2-scored.csv | cut -d',' -f2 | sort -u | wc -l
# Expected: Number of accounts in your organization (e.g., 67)

# List all discovered accounts
tail -n +2 data/outputs/ec2-scored.csv | cut -d',' -f2 | sort -u

Section 3: WHY Unified Taskfile?

The Problem with Dual Taskfiles

Historical Approach (2 separate files): 

Taskfile.inventory.yaml       # Multi-account operations
Taskfile.inventory.single.yaml # Single-account operations

Issues: - ❌ Maintenance burden: 48 task definitions (24 × 2 files) - ❌ DRY violation: Duplicate logic across files - ❌ User confusion: Which file to use? - ❌ Drift risk: Updates to one file missed in the other

The Unified Solution

Single Taskfile with environment variable overrides: 

Taskfile.inventory.yaml        # ALL operations (single + multi-account)

Benefits: - ✅ KISS principle: 1 file instead of 2 (50% reduction) - ✅ DRY principle: 24 task definitions (not 48) - ✅ Flexibility: Profile override support - ✅ Production validated: 67-account LZ operational

Design Philosophy

AWS ~/.aws/config Pattern Compliance:

The unified Taskfile follows the industry-standard AWS configuration pattern:

# AWS CLI pattern (single file with profile selection)
aws s3 ls --profile dev-account          # Single account
aws s3 ls --profile centralised-ops      # Multi-account aggregator

Taskfile pattern (same concept): 

# Override for single account
export CENTRALISED_OPS_PROFILE=my-account
task discover-ec2

# Use defaults for multi-account
task discover-ec2

Maintenance Efficiency Comparison

Aspect Dual Taskfiles Unified Taskfile Improvement
File count 2 files 1 file 50% reduction
Task definitions 48 (24 × 2) 24 50% reduction
Maintenance time 2× updates 1× update 50% faster
Drift risk High Zero Eliminated
User clarity Confusing Clear Improved

Migration Path

From dual Taskfiles: 

# Old approach (deprecated)
task -t Taskfile.inventory.single.yaml discover-ec2  # Single account
task -t Taskfile.inventory.yaml discover-ec2          # Multi-account

To unified Taskfile: 

# New approach (recommended)
export CENTRALISED_OPS_PROFILE=my-account
task -t Taskfile.inventory.yaml discover-ec2          # Single account (override)

unset CENTRALISED_OPS_PROFILE
task -t Taskfile.inventory.yaml discover-ec2          # Multi-account (default)

Section 4: Profile Management Best Practices

4-Profile Architecture

Profile Roles:

Profile Variable AWS Service Purpose Required For
CENTRALISED_OPS_PROFILE Resource Explorer, CloudTrail, CloudWatch, SSM Cross-account resource discovery and activity Layers 1, 4
MANAGEMENT_PROFILE AWS Organizations Account metadata and organizational structure Layer 2
BILLING_PROFILE AWS Cost Explorer Cost data and financial intelligence Layer 3
DEFAULT_PROFILE AWS CLI default Fallback for commands without explicit profile N/A

Auto-Detection Logic

How Taskfile selects profiles:

  1. Check environment variables first 

    export CENTRALISED_OPS_PROFILE="custom-ops-profile"
# Taskfile uses: custom-ops-profile

  2. Use Taskfile defaults if environment not set 

    CENTRALISED_OPS_PROFILE: '{{.CENTRALISED_OPS_PROFILE | default "ams-centralised-ops-ReadOnlyAccess-335083429030"}}'
# Taskfile uses: ams-centralised-ops-ReadOnlyAccess-335083429030

  3. Fail gracefully with clear error messages 

    Error: AWS profile 'my-profile' not found in ~/.aws/config
Action: Run 'aws configure sso --profile my-profile' or check profile name

Profile Configuration Methods

Method 1: Shell Profile (~/.bashrc or ~/.zshrc)

# Add to shell configuration for persistent settings
export MANAGEMENT_PROFILE="org-admin-ReadOnlyAccess-909135376185"
export BILLING_PROFILE="billing-ReadOnlyAccess-909135376185"
export CENTRALISED_OPS_PROFILE="ops-ReadOnlyAccess-335083429030"

Method 2: .env File (project-specific)

# Create .env file in project root
cat > .env << 'EOF'
export MANAGEMENT_PROFILE="your-management-profile"
export BILLING_PROFILE="your-billing-profile"
export CENTRALISED_OPS_PROFILE="your-ops-profile"
EOF

# Add to .gitignore
echo ".env" >> .gitignore

# Load before running tasks
source .env
task -t Taskfile.inventory.yaml pipeline-5-layer

Method 3: Per-Command Override (temporary)

# Override for single command
CENTRALISED_OPS_PROFILE=test-account task discover-ec2

Troubleshooting Profile Issues

Issue 1: Profile Not Found

Error: 

The config profile (my-profile) could not be found

Solution: 

# List available profiles
aws configure list-profiles

# Update environment variable to match
export MANAGEMENT_PROFILE="correct-profile-name"

# Verify profile works
aws sts get-caller-identity --profile $MANAGEMENT_PROFILE

Issue 2: SSO Token Expired

Error: 

An error occurred (ExpiredToken) when calling the DescribeInstances operation

Solution: 

# Re-authenticate with SSO
aws sso login --profile $CENTRALISED_OPS_PROFILE

# Verify token validity
aws sts get-caller-identity --profile $CENTRALISED_OPS_PROFILE

Issue 3: Missing Permissions

Error: 

An error occurred (AccessDenied) when calling the Search operation

Solution: 

# Check required permissions in IAM policy
# Layer 1 requires: resource-explorer-2:Search
# Layer 2 requires: organizations:ListAccounts
# Layer 3 requires: ce:GetCostAndUsage

# Contact AWS administrator to add missing permissions

Security Best Practices

✅ Recommended: - Use AWS SSO profiles (not long-term credentials) - Separate profiles for different API access patterns - Add .env files to .gitignore - Use read-only IAM roles where possible

❌ Avoid: - Hardcoded AWS access keys in scripts - Single admin profile with all permissions - Sharing profiles across environments - Committing credentials to version control

Profile Validation Script

#!/bin/bash
# validate-profiles.sh - Verify all required profiles

echo "🔍 Validating AWS profiles..."

# Check profile variables defined
if [[ -z "$MANAGEMENT_PROFILE" ]]; then
  echo "❌ MANAGEMENT_PROFILE not set"
  exit 1
fi

if [[ -z "$BILLING_PROFILE" ]]; then
  echo "❌ BILLING_PROFILE not set"
  exit 1
fi

if [[ -z "$CENTRALISED_OPS_PROFILE" ]]; then
  echo "❌ CENTRALISED_OPS_PROFILE not set"
  exit 1
fi

# Verify profile access
echo "Checking MANAGEMENT_PROFILE..."
aws sts get-caller-identity --profile $MANAGEMENT_PROFILE > /dev/null || exit 1

echo "Checking BILLING_PROFILE..."
aws sts get-caller-identity --profile $BILLING_PROFILE > /dev/null || exit 1

echo "Checking CENTRALISED_OPS_PROFILE..."
aws sts get-caller-identity --profile $CENTRALISED_OPS_PROFILE > /dev/null || exit 1

echo "✅ All profiles validated successfully"

Usage: 

chmod +x validate-profiles.sh
./validate-profiles.sh

Quick Reference

Single Account Setup

export CENTRALISED_OPS_PROFILE=my-account
export MANAGEMENT_PROFILE=my-account
export BILLING_PROFILE=my-account
task -t Taskfile.inventory.yaml pipeline-5-layer

Multi-Account Setup

# Uses Taskfile defaults (or set your own)
task -t Taskfile.inventory.yaml pipeline-5-layer

Verify Configuration

task -t Taskfile.inventory.yaml show-profiles

Common Commands

# Discovery
task -t Taskfile.inventory.yaml discover-ec2
task -t Taskfile.inventory.yaml discover-rds
task -t Taskfile.inventory.yaml discover-workspaces

# Organizations
task -t Taskfile.inventory.yaml list-accounts
task -t Taskfile.inventory.yaml draw-org

# Complete Pipeline
task -t Taskfile.inventory.yaml pipeline-5-layer
task -t Taskfile.inventory.yaml pipeline-summary