diff --git a/User Manual (IAM role based resource cost breakdown - lambda).pdf b/User Manual (IAM role based resource cost breakdown - lambda).pdf new file mode 100644 index 00000000..b6df39f6 Binary files /dev/null and b/User Manual (IAM role based resource cost breakdown - lambda).pdf differ diff --git a/custom_dashboard/grafana_dashboards/ResourceByServiceAndRole.json b/custom_dashboard/grafana_dashboards/ResourceByServiceAndRole.json new file mode 100644 index 00000000..3189c0ad --- /dev/null +++ b/custom_dashboard/grafana_dashboards/ResourceByServiceAndRole.json @@ -0,0 +1,191 @@ + +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 13, + "links": [], + "panels": [ + { + "datasource": { + "type": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 80, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineWidth": 1, + "scaleDistribution": { + "type": "linear" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 13, + "w": 21, + "x": 0, + "y": 0 + }, + "id": 1, + "options": { + "barRadius": 0, + "barWidth": 0.97, + "fullHighlight": false, + "groupWidth": 0.7, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "orientation": "auto", + "showValue": "auto", + "stacking": "none", + "tooltip": { + "mode": "single", + "sort": "none" + }, + "xField": "iam_role_service_resource_name", + "xTickLabelRotation": 0, + "xTickLabelSpacing": 0 + }, + "targets": [ + { + "datasource": { + "type": "prometheus" + }, + "disableTextWrap": false, + "editorMode": "builder", + "expr": "IAM_Role_Service{iam_role_of_service=\"$role\", iam_role_service_name=\"$Service\"}", + "format": "table", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "legendFormat": "", + "range": true, + "refId": "A", + "useBackend": false + } + ], + "title": "Resource Cost", + "type": "barchart" + } + ], + "schemaVersion": 39, + "tags": [], + "templating": { + "list": [ + { + "current": { + "selected": false, + "text": "test-role", + "value": "test-role" + }, + "datasource": { + "type": "prometheus" + }, + "definition": "query_result(IAM_Role_Service)", + "hide": 0, + "includeAll": false, + "multi": false, + "name": "role", + "options": [], + "query": { + "qryType": 3, + "query": "query_result(IAM_Role_Service)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "/iam_role_of_service=\"(?[^\"]+)/", + "skipUrlSync": false, + "sort": 0, + "type": "query" + }, + { + "current": { + "selected": false, + "text": "lambda", + "value": "lambda" + }, + "datasource": { + "type": "prometheus" + }, + "definition": "query_result(IAM_Role_Service)", + "hide": 0, + "includeAll": false, + "multi": false, + "name": "Service", + "options": [], + "query": { + "qryType": 3, + "query": "query_result(IAM_Role_Service)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "/iam_role_service_name=\"(?[^\"]+)/", + "skipUrlSync": false, + "sort": 0, + "type": "query" + } + ] + }, + "time": { + "from": "now", + "to": "now" + }, + "timepicker": {}, + "timezone": "browser", + "title": "Resource by Role and Service", + "uid": "ddilv8psdglxca", + "version": 2, + "weekStart": "" +} diff --git a/custom_dashboard/grafana_dashboards/Services_by_iam_role.json b/custom_dashboard/grafana_dashboards/Services_by_iam_role.json new file mode 100644 index 00000000..712ab662 --- /dev/null +++ b/custom_dashboard/grafana_dashboards/Services_by_iam_role.json @@ -0,0 +1,268 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 11, + "links": [], + "panels": [ + { + "datasource": { + "type": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 80, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineWidth": 1, + "scaleDistribution": { + "type": "linear" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [ + { + "targetBlank": false, + "title": "Resources", + "url": "/d/ddilv8psdglxca/resource-by-role-and-service?orgId=1&var-Service=${__data.fields.iam_role_service_name}&var-role=${Role}" + } + ], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 15, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 2, + "options": { + "barRadius": 0, + "barWidth": 0.97, + "fullHighlight": false, + "groupWidth": 0.7, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "orientation": "auto", + "showValue": "auto", + "stacking": "none", + "tooltip": { + "mode": "single", + "sort": "none" + }, + "xField": "iam_role_service_name", + "xTickLabelRotation": 0, + "xTickLabelSpacing": 0 + }, + "targets": [ + { + "datasource": { + "type": "prometheus" + }, + "disableTextWrap": false, + "editorMode": "builder", + "expr": "sum by(iam_role_service_name) (topk(5, IAM_Role_Service{iam_role_of_service=\"$Role\"}))", + "format": "table", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "legendFormat": "", + "range": true, + "refId": "A", + "useBackend": false + } + ], + "title": "Cost by Services", + "type": "barchart" + }, + { + "datasource": { + "type": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "fillOpacity": 80, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineWidth": 1, + "scaleDistribution": { + "type": "linear" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 13, + "w": 17, + "x": 0, + "y": 15 + }, + "id": 1, + "options": { + "barRadius": 0, + "barWidth": 0.97, + "fullHighlight": false, + "groupWidth": 0.7, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "orientation": "auto", + "showValue": "auto", + "stacking": "none", + "tooltip": { + "mode": "single", + "sort": "none" + }, + "xField": "iam_role_service_resource_name", + "xTickLabelRotation": 0, + "xTickLabelSpacing": 0 + }, + "targets": [ + { + "datasource": { + "type": "prometheus" + }, + "disableTextWrap": false, + "editorMode": "builder", + "expr": "topk(5, IAM_Role_Service{iam_role_of_service=\"$Role\"})", + "format": "table", + "fullMetaSearch": false, + "includeNullMetadata": false, + "instant": false, + "legendFormat": "__auto", + "range": true, + "refId": "A", + "useBackend": false + } + ], + "title": "Cost by resources", + "type": "barchart" + } + ], + "schemaVersion": 39, + "tags": [], + "templating": { + "list": [ + { + "current": { + "selected": false, + "text": "test-role", + "value": "test-role" + }, + "datasource": { + "type": "prometheus" + }, + "definition": "query_result(IAM_Role_Service)", + "hide": 0, + "includeAll": false, + "multi": false, + "name": "Role", + "options": [], + "query": { + "qryType": 3, + "query": "query_result(IAM_Role_Service)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "/iam_role_of_service=\"(?[^\"]+)/", + "skipUrlSync": false, + "sort": 0, + "type": "query" + } + ] + }, + "time": { + "from": "now", + "to": "now" + }, + "timepicker": {}, + "timezone": "browser", + "title": "Resource by Role", + "uid": "fdhtgs5qdvny8f", + "version": 7, + "weekStart": "" +} \ No newline at end of file diff --git a/custom_dashboard/grafana_dashboards/grafana-dashboard.md b/custom_dashboard/grafana_dashboards/grafana-dashboard.md index 99a58e8a..5bd29f75 100644 --- a/custom_dashboard/grafana_dashboards/grafana-dashboard.md +++ b/custom_dashboard/grafana_dashboards/grafana-dashboard.md @@ -6,9 +6,10 @@ We have a total of 4 Grafana Dashboards. The home dashboard consists of the following panels: - Tracking Total Costs Across Multiple Accounts -- A specific account cost for he last 3 months +- A specific account cost for the last 3 months - Project Spend cost - Top 5 most expensive services +- Top 5 most expensive roles ## **Associated Service Dashboard** @@ -29,6 +30,15 @@ The IAM Role Dashboard consists of the following panels: - Resource Cost (Last 14 days) - Start/Stop Action +## **Service By Role Dashboard** +- Service details (last 14 days) +- Resource Details (last 14 days) +- To5 5 service and resource filtered by a role + +## ## **Resource By Role and Service Dashboard** +- Resource Details (last 14 days) +- To5 5 resources filtered by role and service + ## **IAM User Dashboard** The IAM User Dashboard consists of the following panels: diff --git a/custom_dashboard/grafana_dashboards/home-dashboard.json b/custom_dashboard/grafana_dashboards/home-dashboard.json index 1ece0e69..4f7abf2e 100644 --- a/custom_dashboard/grafana_dashboards/home-dashboard.json +++ b/custom_dashboard/grafana_dashboards/home-dashboard.json @@ -340,6 +340,115 @@ "mode": "palette-classic" }, "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "series", + "axisLabel": "Cost", + "axisPlacement": "auto", + "axisWidth": 40, + "fillOpacity": 80, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineWidth": 3, + "scaleDistribution": { + "type": "linear" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [ + { + "targetBlank": false, + "title": "Cost by Role", + "url": "/d/fdhtgs5qdvny8f/resource-by-role?orgId=1&var-Role=${__data.fields.iam_role_of_service}" + } + ], + "mappings": [], + "noValue": "0", + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "none" + }, + "overrides": [] + }, + "gridPos": { + "h": 14, + "w": 24, + "x": 0, + "y": 19 + }, + "id": 13, + "options": { + "barRadius": 0, + "barWidth": 0.97, + "fullHighlight": false, + "groupWidth": 0.7, + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "orientation": "auto", + "showValue": "auto", + "stacking": "none", + "tooltip": { + "mode": "single", + "sort": "none" + }, + "xField": "iam_role_of_service", + "xTickLabelMaxLength": 35, + "xTickLabelRotation": 0, + "xTickLabelSpacing": 100 + }, + "targets": [ + { + "datasource": { + "type": "prometheus" + }, + "disableTextWrap": false, + "editorMode": "builder", + "expr": "sum by(iam_role_of_service) (topk(5, IAM_Role_Service))", + "format": "table", + "fullMetaSearch": false, + "includeNullMetadata": true, + "instant": false, + "legendFormat": "", + "range": true, + "refId": "A", + "useBackend": false + } + ], + "title": "Top 5 Most Expensive Roles", + "type": "barchart" + }, + { + "datasource": { + "type": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", diff --git a/infrastructure/main.tf b/infrastructure/main.tf index c52e1df1..04bf6ddb 100644 --- a/infrastructure/main.tf +++ b/infrastructure/main.tf @@ -83,4 +83,5 @@ module "serverless" { create_cloudtrail = var.create_cloudtrail create_cloudtrail_s3_bucket = var.create_cloudtrail_s3_bucket env = var.env + report_bucket_name = var.report_bucket_name } diff --git a/infrastructure/modules/serverless/iam-role.tf b/infrastructure/modules/serverless/iam-role.tf index 5e3b0dfe..a1de885b 100644 --- a/infrastructure/modules/serverless/iam-role.tf +++ b/infrastructure/modules/serverless/iam-role.tf @@ -171,7 +171,8 @@ resource "aws_iam_role_policy" "IamRolesServiceMapping" { "Effect" : "Allow", "Action" : [ "iam:ListInstanceProfilesForRole", - "iam:PassRole" + "iam:PassRole", + "lambda:ListFunctions" ], "Resource" : ["*"] }, @@ -215,6 +216,7 @@ resource "aws_lambda_function" "IamRolesService" { variables = { prometheus_ip = "${var.prometheus_ip}:9091" region_names_path = "/${var.namespace}/region_names" + report_bucket_name = var.report_bucket_name } } memory_size = var.memory_size @@ -223,8 +225,8 @@ resource "aws_lambda_function" "IamRolesService" { subnet_ids = [var.subnet_id[0]] security_group_ids = [var.security_group_id] } - - layers = [var.prometheus_layer] + // layer arn for pandas + layers = [var.prometheus_layer, "arn:aws:lambda:ap-southeast-2:336392948345:layer:AWSSDKPandas-Python39:18"] tags = merge(local.tags, tomap({ "Name" = "${var.namespace}-iamroleservice" })) @@ -271,6 +273,15 @@ resource "aws_iam_role_policy" "IamRolesService" { "ec2:AttachNetworkInterface" ], "Resource" : "*" + }, + { + "Effect" : "Allow", + "Action" : [ + "s3:GetObject" + ], + "Resource" : [ + "arn:aws:s3:::${var.report_bucket_name}/*" + ] } ] }) diff --git a/infrastructure/modules/serverless/variables.tf b/infrastructure/modules/serverless/variables.tf index 623bc392..7f038172 100644 --- a/infrastructure/modules/serverless/variables.tf +++ b/infrastructure/modules/serverless/variables.tf @@ -144,3 +144,7 @@ variable "env" { type = string } +variable "report_bucket_name" { + type = string + description = "The name of the bucket containing the Cost and Usage Report" +} \ No newline at end of file diff --git a/infrastructure/terraform.auto.tfvars b/infrastructure/terraform.auto.tfvars index fe68b285..d0540bd1 100644 --- a/infrastructure/terraform.auto.tfvars +++ b/infrastructure/terraform.auto.tfvars @@ -42,6 +42,7 @@ project = "testing" create_cloudtrail_kms = true create_cloudtrail = true create_cloudtrail_s3_bucket = true +report_bucket_name = "report-bucket" // bucket name which contains Cost and Usage Report security_group_ingress = { "pushgateway" = { description = "PushGateway" diff --git a/infrastructure/variables.tf b/infrastructure/variables.tf index fd9738f6..15f5344b 100644 --- a/infrastructure/variables.tf +++ b/infrastructure/variables.tf @@ -158,3 +158,8 @@ variable "env" { type = string default = "dev" } + +variable "report_bucket_name" { + type = string + description = "The name of the bucket containing the Cost and Usage Report" +} \ No newline at end of file diff --git a/init.sh b/init.sh index d7d30204..678404a1 100644 --- a/init.sh +++ b/init.sh @@ -343,7 +343,7 @@ done echo "Changing Directory: $(pwd)" cd infrastructure || { echo "Failure : cd infrastructure not working"; exit 1; } mkdir python -pip3 install -t python/ prometheus-client +pip3 install -t python/ prometheus-client pandas==2.2 numpy==1.22.4 pytz==2020.1 python-dateutil==2.8.2 tzdata==2022.7 zip -r python.zip ./python echo "Packaging Prometheus Done" diff --git a/src/iam_roles/iam_roles_all.py b/src/iam_roles/iam_roles_all.py index 9d918a7d..56265ea1 100644 --- a/src/iam_roles/iam_roles_all.py +++ b/src/iam_roles/iam_roles_all.py @@ -56,6 +56,7 @@ def get_region_names(): # Get the region names dictionary region_names = get_region_names() + def lambda_handler(event, context): """ Lambda handler function that extracts IAM role information diff --git a/src/iam_roles/iamrolesservice.py b/src/iam_roles/iamrolesservice.py index bc514123..bd65ed80 100644 --- a/src/iam_roles/iamrolesservice.py +++ b/src/iam_roles/iamrolesservice.py @@ -12,15 +12,22 @@ # See the License for the specific language governing permissions and # limitations under the License. +import gzip +import io import json import logging import os from datetime import datetime, timedelta +from dateutil import relativedelta import boto3 from prometheus_client import CollectorRegistry, Gauge, push_to_gateway +import pandas as pd - +try: + s3 = boto3.client("s3") +except Exception as e: + logging.error("Error creating boto3 client: " + str(e)) try: client = boto3.client("ce") except Exception as e: @@ -89,6 +96,54 @@ def get_region_names(): # Get the region names dictionary region_names = get_region_names() + +def get_cur_data(): + bucket = os.environ["report_bucket_name"] + + current_date = datetime.now() + start_date_str = f"{current_date.strftime('%Y%m')}01" + end_date = datetime.strptime(start_date_str, "%Y%m%d") + relativedelta.relativedelta(months=1) + end_date_str = f"{end_date.strftime('%Y%m')}01" + full_date_range = f"{start_date_str}-{end_date_str}" + + key = f"report/reportbucket/{full_date_range}/reportbucket-00001.csv.gz" + try: + response = s3.get_object(Bucket=bucket, Key=key) + resource_file = response["Body"].read() + cur_data = [] + with gzip.GzipFile(fileobj=io.BytesIO(resource_file), mode="rb") as data: + cur_data = pd.read_csv(io.BytesIO(data.read())) + cur_data = cur_data[[ + "product/ProductName", + "lineItem/ResourceId", + "lineItem/UnblendedCost", + ]] + return cur_data + except Exception as e: + logging.error( + "Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.".format( + key, bucket + ) + ) + raise + +# Get the cost and usage report data +cur_data = get_cur_data() + +def get_cumulative_cost(resource_id): + """ + Fetches the cost for a given resource ID from the Cost and Usage Report. + + Args: + resource_id (str): The ID of the resource to fetch the cost for. + + Returns: + The cumulative cost of the resource in the Cost and Usage Report. + """ + df = cur_data + cost = df.loc[df["lineItem/ResourceId"] == resource_id].sum()["lineItem/UnblendedCost"] + return cost + def lambda_handler(event, context): """ The main Lambda function that is executed. @@ -110,9 +165,11 @@ def lambda_handler(event, context): labelnames=[ "TimeSlot", "iam_role_of_service", + "iam_role_service_name", "iam_role_service_region", "iam_role_service_account", "iam_role_service_resource_id", + "iam_role_service_resource_name", "iam_role_service_cost", "iam_role_service_state", ], @@ -150,9 +207,11 @@ def lambda_handler(event, context): "%Y-%m-%d %H:%M:%S" ), role, + "None", role_region, account_id, "None", + "None", "0", "Start", ).set(0) @@ -161,73 +220,101 @@ def lambda_handler(event, context): # check if detail is a dictionary if isinstance(detail, dict): - # extract the "Instance_Region" and "Instance" fields - instance_region = detail["Instance_Region"] - instance = detail["Instance"] - ec2 = "ec2:instance/" + instance - response = cost_of_instance( - event, client, instance, start_date, end_date - ) + if detail["Service"] == "ec2": + # extract the "Instance_Region" and "Instance" fields + instance_region = detail["Instance_Region"] + instance = detail["Instance"] + ec2 = "ec2:instance/" + instance + response = cost_of_instance( + event, client, instance, start_date, end_date + ) - ec2_resource = boto3.resource("ec2", region_name=instance_region) - state = ec2_resource.Instance(instance).state["Name"] - - if state != "terminated": - if state == "running": - - cumulative = 0.0 - for j in range(len(response["ResultsByTime"])): - time_Data = response["ResultsByTime"][j]["TimePeriod"][ - "End" - ] - new_time = time_Data.replace("00:00:00", "12:02:02") - cumulative = cumulative + float( - response["ResultsByTime"][j]["Total"][ - "UnblendedCost" - ]["Amount"] - ) + ec2_resource = boto3.resource("ec2", region_name=instance_region) + state = ec2_resource.Instance(instance).state["Name"] - iam_service_gauge.labels( - ( - datetime.strptime( - new_time, "%Y-%m-%dT%H:%M:%SZ" - ) - ).strftime("%Y-%m-%d %H:%M:%S"), - role, - f"{role_region} ({region_names.get(role_region, 'unknown region name')})", - account_id, - ec2, - cumulative, - "Stop", - ).set(cumulative) - elif state == "stopped": - cumulative = 0.0 - for j in range(len(response["ResultsByTime"])): - time_Data = response["ResultsByTime"][j]["TimePeriod"][ - "End" - ] - new_time = time_Data.replace("00:00:00", "12:02:02") - cumulative = cumulative + float( - response["ResultsByTime"][j]["Total"][ - "UnblendedCost" - ]["Amount"] - ) + if state != "terminated": + if state == "running": + + cumulative = 0.0 + for j in range(len(response["ResultsByTime"])): + time_Data = response["ResultsByTime"][j]["TimePeriod"][ + "End" + ] + new_time = time_Data.replace("00:00:00", "12:02:02") + cumulative = cumulative + float( + response["ResultsByTime"][j]["Total"][ + "UnblendedCost" + ]["Amount"] + ) + + iam_service_gauge.labels( + ( + datetime.strptime( + new_time, "%Y-%m-%dT%H:%M:%SZ" + ) + ).strftime("%Y-%m-%d %H:%M:%S"), + role, + detail["Service"], + f"{role_region} ({region_names.get(role_region, 'unknown region name')})", + account_id, + ec2, + ec2, + cumulative, + "Stop", + ).set(cumulative) + elif state == "stopped": + cumulative = 0.0 + for j in range(len(response["ResultsByTime"])): + time_Data = response["ResultsByTime"][j]["TimePeriod"][ + "End" + ] + new_time = time_Data.replace("00:00:00", "12:02:02") + cumulative = cumulative + float( + response["ResultsByTime"][j]["Total"][ + "UnblendedCost" + ]["Amount"] + ) - iam_service_gauge.labels( - ( - datetime.strptime( - new_time, "%Y-%m-%dT%H:%M:%SZ" - ) - ).strftime("%Y-%m-%d %H:%M:%S"), - role, - f"{role_region} ({region_names.get(role_region, 'unknown region name')})", - account_id, - ec2, - cumulative, - "Start", - ).set(cumulative) - else: - continue + iam_service_gauge.labels( + ( + datetime.strptime( + new_time, "%Y-%m-%dT%H:%M:%SZ" + ) + ).strftime("%Y-%m-%d %H:%M:%S"), + role, + detail["Service"], + f"{role_region} ({region_names.get(role_region, 'unknown region name')})", + account_id, + ec2, + ec2, + cumulative, + "Start", + ).set(cumulative) + else: + continue + + elif detail["Service"] == "lambda": + # extract the "Function" field + function = detail["Function"] + lambda_function = "lambda:function/" + function + lambda_function_name = "lambda:function/" + function.split(":")[-1] + lambda_cost = get_cumulative_cost(function) + + iam_service_gauge.labels( + ( + datetime.strptime( + new_time, "%Y-%m-%dT%H:%M:%SZ" + ) + ).strftime("%Y-%m-%d %H:%M:%S"), + role, + detail["Service"], + f"{role_region} ({region_names.get(role_region, 'unknown region name')})", + account_id, + lambda_function, + lambda_function_name, + lambda_cost, + "None", + ).set(lambda_cost) elif isinstance(detail, str): iam_service_gauge.labels( @@ -235,9 +322,11 @@ def lambda_handler(event, context): "%Y-%m-%d %H:%M:%S" ), role, + "None", f"{role_region} ({region_names.get(role_region, 'unknown region name')})", account_id, detail, + detail, "0", "Start", ).set(0) diff --git a/src/iam_roles/iamrolesservicemapping.py b/src/iam_roles/iamrolesservicemapping.py index eeefc53e..99c149e0 100644 --- a/src/iam_roles/iamrolesservicemapping.py +++ b/src/iam_roles/iamrolesservicemapping.py @@ -125,10 +125,44 @@ def lambda_handler(event, context): "AvailabilityZone" ][:-1] instance_detail = { + "Service": "ec2", "Instance_Region": instance_region, "Instance": instance_id, } service_mapping.append(instance_detail) + elif resource == "lambda": + if role_region == "None": + # role is not in use + continue + else: + service_client = boto3.client( + resource + ) + try: + list_of_lambdas = service_client.list_functions() + except Exception as e: + logging.error("Error getting list of lambdas" + str(e)) + return { + "statusCode": 500, + "body": json.dumps({"Error": str(e)}), + } + + # getting lambdas description from the above list + lambdas_iterator = list_of_lambdas["Functions"] + for function in lambdas_iterator: + function_arn = function["FunctionArn"] + function_region = function["FunctionArn"].split(':')[3] + function_role_arn = function["Role"] + if function_role_arn != role_arn: + # if the lambda is not assuming this role + continue + else: + function_detail = { + "Service": "lambda", + "Function_Region": function_region, + "Function": function_arn + } + service_mapping.append(function_detail) else: # adding other services service_mapping.append(resource)