🚀 AWS Cloud Asset Inventory Toolkit¶

🎓 Enterprise-Grade Production-Ready Automation for AWS Resource Inventory, Management, and Governance

📖 Overview¶

The AWS Cloud Asset Inventory Toolkit provides comprehensive Python-based automation scripts specifically designed for large-scale, multi-account AWS environments. These scripts simplify asset discovery, compliance audits, security enforcement, workload management, and governance aligned with AWS Cloud Foundation, AWS Well-Architected Framework, CIS Controls, NIST SP 800-53, and ISO 27001 best practices.

This repository adheres strictly to industry-leading DevSecOps principles:

Discover every AWS asset across 1…N Organizations, Accounts and Regions.
Security First: Zero Trust architecture and secure-by-default scripting.
Automation & Repeatability: Infrastructure-as-Code (IaC) inspired automation and script standardization.
Observability & Auditing: Built-in structured logging and audit trails for governance and compliance visibility.
Scalability & Maintainability: Clearly defined script naming conventions and documentation, designed for future scalability and ease of maintenance.

🏗️ AWS Cloud Foundation Capability Matrix

Capability Area	Typical Questions Solved	Representative Scripts^‡
Identity & Access Control	Which roles / SAML IdPs exist? Do they comply with policy X?	`list_iam_roles.py`, `list_iam_saml_providers.py`, `UpdateRoleToMemberAccounts.py`
Log Storage	Is CloudTrail enabled everywhere? Are log-groups retained 365 d?	`check_cloudtrail_status.py`, `update_cloudwatch_retention_policy.py`, `summarize_vpc_flow_logs.py`
Governance & Policy Enforcement	Are S3 buckets publicly blocked? Are CT prerequisites met?	`enforce_s3_public_access_block.py`, `check_organizations_ct_prereqs.py`, `draw_organizations_structure.py`
Workload Isolation	List all compute & storage assets per account/OU.	`list_ec2_instances.py`, `list_lambda_functions.py`, `list_ecs_clusters_and_tasks.py`
Network Connectivity	Find all VPCs, subnets, ENIs, overly-permissive SGs.	`list_ec2_vpcs.py`, `find_ec2_network_interfaces.py`, `verify_ec2_security_groups.py`
Change Management	Detect drift, migrate StackSets, clean orphaned stacks.	`enable_cfn_stackset_drift_detection.py`, `move_cfn_stackset_instances.py`, `list_cfn_orphaned_stacks.py`
Cloud Financial Management	Which EBS volumes are unattached? How much can we save?	`list_ec2_ebs_volumes.py`, `summarize_vpc_flow_logs.py`
Tag Governance (road-map)	–	–

🗂️ Complete Script Catalogue/Structure¶

📌 Script Naming Convention & 🛠️ AWS Capability Alignment

Each script aligns to specific AWS Cloud Foundation capabilities and follows a clear naming convention:

<verb>_<domain>_<object>.py

Example: list_ec2_instances.py

Verb–Domain–Object structure:

Verb = Action > what it does (e.g. list, check, enable, enforce, update, delete, modify, map, summarize, draw, recover, run).
Domain = AWS Service/Domain > which AWS service or concept (e.g. ec2, iam, cfn, organizations, s3, cloudtrail, vpc, …).
Object = Specific the resource type or logical entity targeted (e.g. instances, roles, stacksets, public_access_block, …).

Mutating Actions: Scripts that change infrastructure use verbs enforce, update, delete, modify and require an explicit +apply / +delete flag. By default every tool is read-only.

The table below lists every operational script in the toolkit, grouped by capability.

Capability	Script (new name)	Brief Purpose
🔐 Identity & Access Control
	`list_iam_roles.py`	Lists all IAM roles across AWS accounts.
	`list_organizations_iam_users.py`	Enumerate IAM users across Organizations.
	`list_iam_saml_providers.py`	Discovers all configured SAML IdPs provider; optional safe delete.
	`list_iam_policies.py`	Inventory customer-managed & inline policies.
	`UpdateRoleToMemberAccounts.py`	Convert ALZ role trusts to CT model.
📈 Logging & Observability
	`check_cloudtrail_status.py`	Audits CloudTrail Org/Account enablement status in all regions.
	`update_cloudwatch_retention_policy.py`	Updates retention settings across CloudWatch logs & estimate savings.
	`summarize_vpc_flow_logs.py`	Aggregate & cost-analyse VPC Flow Logs.
	`list_config_recorders_delivery_channels.py`	Lists AWS Config recorders and delivery channels.
🛡 Governance & Compliance & Policy Enforcement
	`enforce_s3_public_access_block.py`	Apply org-wide S3 Public-Block controls.
	`check_organizations_alz_prereqs.py`	Validates ALZ prerequisites for AWS accounts; optional remediation.
	`check_organizations_ct_prereqs.py`	Checks prerequisites for AWS Control Tower adoption.
	`draw_organizations_structure.py`	Generates AWS Org diagrams in Graphviz format of OUs / accounts / SCPs.
	`map_servicecatalog_products_to_cfn_stacks.py`	Reconcile SC products ↔ CFN stacks; clean errors.
	`lock_down_cfn_stacksets_role.py`	Harden/unharden IAM role used by StackSets.
🚧 Workload Isolation
	`list_ec2_instances.py`	Org-wide inventory of EC2 instances.
	`list_ec2_ebs_volumes.py`	Find unattached / idle EBS volumes.
	`list_lambda_functions.py`	List Lambda functions; optional runtime update.
	`list_ecs_clusters_and_tasks.py`	Inventory ECS clusters, services & tasks.
	`list_ds_directories.py`	Discover AWS Directory Service directories.
	`list_sns_topics.py`	Enumerate SNS topics.
🌐 Network Connectivity
	`list_ec2_vpcs.py`	Lists VPCs across accounts and regions.
	`list_ec2_subnets.py`	Enumerates subnets, searchable by reverse IP lookup.
	`find_ec2_network_interfaces.py`	Locate ENIs by IP address.
	`find_ec2_security_groups.py`	Locate SGs by ID / name / CIDR.
	`verify_ec2_security_groups.py`	Detect overly-permissive rules.
	`list_route53_private_hosted_zones.py`	Lists Route53 private hosted zones.
	`list_ec2_availability_zones.py`	Show enabled AZs per region/account.
🔄 Change Management
	`list_cfn_stacksets.py`	List CloudFormation StackSets across accounts.
	`enable_cfn_stackset_drift_detection.py`	Enables drift detection for CloudFormation StackSets.
	`list_cfn_orphaned_stacks.py`	Identifies orphaned child stacks.
	`move_cfn_stackset_instances.py`	Safely migrates CloudFormation stack instances across StackSets.
	`modify_cfn_stacksets.py`	Remove closed accounts, re-run templates.
	`list_cfn_stacks.py`	Search stacks across accounts/regions.
	`list_cfn_stackset_operations.py` ?	Summarise last operation per StackSet.
	`recover_cfn_stack_ids.py`	Rebuild StackID → account mapping.
	`enable_cfn_drift_detection.py`	Generic per-stack drift detection.
	`delete_landing_zone.py`	Tear-down ALZ resources safely.
	`delete_s3_buckets_and_objects.py`	Bulk delete S3 buckets/objects (alpha).
	`run_commands_on_accounts.py`	Parallel runner for ad-hoc boto3 commands.
	`find_landing_zone_versions.py`	Report ALZ version in mgmt accounts.
Cloud Financial Management
	`list_ec2_ebs_volumes.py`	(see above)
	`summarize_vpc_flow_logs.py`	(see above)

Utilities & libraries: Inventory_Modules.py, account_class.py, ArgumentsClass.py, vpc_modules.py, plus 50 + unit tests under tests/.

⚙️ Quick Start & Usage Guide¶

✅ Pre-requisites¶

AWS CLI and Python (3.10+) installed
AWS credentials with appropriate permissions (stored securely)
Install required Python packages:
```
pip install -r requirements.txt
```

⚙️ Usage (Running Scripts) 🚀¶

Run scripts individually or batch operations:

./inventory_scripts.sh <script_name|all> [parameters]

echo "Execute a specific script:"
./inventory_scripts.sh list_ec2_instances.py -p <profile> -r <region>

echo "Execute all scripts:"
./inventory_scripts.sh all -p <profile>

## Dry-run every non-interactive script
./runbooks/inventory/inventory_scripts.sh all -p org-mgmt -r all

## Audit EC2 instances in a single region
python runbooks/inventory/list_ec2_instances.py -p org-mgmt -r ap-southeast-2 > ec2.csv

[ ] Logs are stored in structured directories (logs/) with detailed execution summaries: test_output_<script>.<timestamp>.log and a global test_output_summary.<timestamp>.txt.

📃 Common CLI-Script Parameters¶

Flag	Purpose	Notes
`-p`	AWS CLI profile (fragments allowed)	Accepts multiple space-delimited profiles.
`-r`	AW Region (`all` = every enabled region)	Defaults to `us-east-1`.
`-v/-vv/-vvv`	INFO / DEBUG verbosity-levels	`-d` alias retained for backward compatibility.
`--exact`	Match exact string (no fragments)	Stacks, profiles.
`--skipprofile`, `--skipaccount`	Exclude items	Space-delimited list.
`--filename`	CSV-friendly output path	Many inventory scripts support.
`+delete`, `+apply`	Explicitly Required flag for deletion actions - for destructive ops	Prevents accidental impact.

[ ] Every script implements -h/--help via argparse, rendering a consistent usage header.

🚩 Security & Operational Guidelines¶

Scripts default to read-only operations unless explicitly stated.
Explicit Actions: Deletion operations require explicit confirmation (+delete parameter) to minimize accidental impact.
Structured logging (JSON format recommended) ensures auditability and traceability + advanced markdown for user-friendly.

📚 References & Documentation¶

AWS Well-Architected Framework
📑 Compliance & Standards Alignment
- CIS AWS Foundations Benchmark
- NIST SP 800-53 Security Controls
- ISO/IEC 27001 Information Security Standards