Initial commit of code, scripts and guide

Author: andkamel

README.md

@@ -1,17 +1,99 @@
-## My Project
+# aws-config-detect-environment-variables-secrets
 
-TODO: Fill this README out!
+Code to deploy a solution to detect secrets/tokens in Lambda functions using AWS Config.
 
-Be sure to:
+## Table of Contents
+
+- [Description](#description)
+- [Prerequisites](#prerequisites)
+- [Dependencies](#dependencies)
+- [Variables](#variables)
+- [Usage](#usage)
+    - [Preparing the Lambda layers (Bash Script)](#preparing-the-lambda-layers-bash-script)
+    - [Deploying the code (Terraforms)](#deploying-the-code-terraforms)
+- [Security](#security)
+- [License](#license)
+
+
+## Description
+
+This Terraform module create a custom rule on AWS Config that detects secrets/tokens in the Lambda functions in the account.
+
+The following diagram applies to the current solution.
+
+![Diagram](.images/lambda-secrets-detector.png)
+
+Once a secret/token is identified in the environment variables of a Lambda function, they are flagged as NON_COMPLIANT with an annotation showing the type of the detected secret. (Example: _JSON Web Token_)
+The AWS Config rule is triggered at any modification of every environment variable in each Lambda functions in the account.
+
+> Pay attention:
+this module is meant to be used as standalone module.
+
+## Prerequisites
+ 
+* **AWS Config**: 
+This module expects that [AWS Config](https://aws.amazon.com/config/) is already up and running in the region where the rules will be deployed.
+The setup can be easily carried out by following the official [documentation](https://docs.aws.amazon.com/config/latest/developerguide/setting-up-aws-config-rules-with-console.html).
+
+* **Docker**:
+In order to properly deploy the resources to your account, Docker needs to be installed on your machine. Please refer to this [link](https://docs.docker.com/get-docker/).
+
+## Dependencies
+
+* **rdklib**: 0.2.0 [Reference](https://github.com/awslabs/aws-config-rdklib);
+* **detect-secrets**: 1.1.0 [Reference](https://github.com/Yelp/detect-secrets)
+
+## Variables
+
+The available variables are described in [variables.tf](./variables.tf) file.
+
+## Usage
+
+In this example we are going to deploy the custom rule that will automatically check all lambda functions for vulnerable environment variables.
+
+### Preparing the Lambda layers (Bash Script)
+
+The lambda function that will be leveraged to carry out the detection process uses two libraries [rdklib](https://github.com/awslabs/aws-config-rdklib) and [detect-secrets](https://github.com/Yelp/detect-secrets)
+Therefore, you need to first ensure those libraries are available to the function in your account as lambda layers.
+
+Use this command in your terminal to prepare them for Terraforms to deploy them in the following section:
+
+```bash
+cd scripts
+./build.sh
+```
+
+### Deploying the code (Terraforms)
+
+**Option 1:**
+You can use the following sample to utilize the module within your code:
+
+```yaml
+module "deploy_lambda_secrets_detector_rule" {
+  config_rule_name = "lambda_has_no_secrets"
+  source           = "./modules/lambda_has_no_secrets"
+  enabled          = true
+}
+```
+Please have a look inside inside [variables.tf](./variables.tf) for all the possible options.
+
+**Option 2:**
+Alternatively, if you have [Terraform](https://www.terraform.io/) installed on your workstation, you can deploy the example by executing:
+
+```bash
+export AWS_PROFILE=$MY_AWS_PROFILE
+terraform init
+terraform apply
+```
+
+> Pay attention:
+you should first modify the `region` in your AWS Profile in accordance to your requirements.
 
-* Change the title in this README
-* Edit your repository description on GitHub
 
 ## Security
 
-See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
+See CONTRIBUTING for more information.
 
 ## License
 
-This project is licensed under the Apache-2.0 License.
-
+This project is licensed under the Apache-2.0 License.

lambda.tf

@@ -0,0 +1,60 @@
+module "lambda_function" {
+  create        = var.enabled
+  source        = "terraform-aws-modules/lambda/aws"
+  function_name = var.config_rule_name
+  description   = "Used by AWS Config to check for secret keys in lambda functions"
+  handler       = "lambda_has_no_secrets.lambda_handler"
+  runtime       = "python3.8"
+  timeout       = 10
+  publish       = true
+  source_path   = "${path.module}/src/lambda_has_no_secrets.py"
+  allowed_triggers = {
+    AllowExecutionFromConfig = {
+      principal = "config.amazonaws.com"
+    }
+  }
+  layers = [
+    module.rdklib_layer.lambda_layer_arn,
+    module.detectsecretslib_layer.lambda_layer_arn
+  ]
+  attach_policy = true
+  policy        = "arn:aws:iam::aws:policy/AWSLambda_ReadOnlyAccess"
+
+  attach_policy_statements = true
+  policy_statements = {
+    AWSConfig_AssumeRole = {
+      effect    = "Allow",
+      actions   = ["sts:AssumeRole"],
+      resources = ["arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/${var.config_rule_name}"]
+    },
+    AWSConfig_PutEvaluations = {
+      effect    = "Allow",
+      actions   = ["config:PutEvaluations"],
+      resources = ["*"]
+    }
+  }
+}
+
+module "rdklib_layer" {
+  create                 = var.enabled
+  source                 = "terraform-aws-modules/lambda/aws"
+  create_layer           = true
+  layer_name             = "rdklib-layer"
+  description            = "RDK Library Layer"
+  compatible_runtimes    = ["python3.8"]
+  create_package         = false
+  local_existing_package = "${path.module}/src/layers/rdklib-layer.zip"
+  create_role            = false
+}
+
+module "detectsecretslib_layer" {
+  create                 = var.enabled
+  source                 = "terraform-aws-modules/lambda/aws"
+  create_layer           = true
+  layer_name             = "detectsecretslib-layer"
+  description            = "Detect Secrets Library Layer"
+  compatible_runtimes    = ["python3.8"]
+  create_package         = false
+  local_existing_package = "${path.module}/src/layers/detect-secrets-layer.zip"
+  create_role            = false
+}

main.tf

@@ -0,0 +1,21 @@
+data "aws_region" "current" {}
+data "aws_caller_identity" "current" {}
+
+resource "aws_config_config_rule" "lambda_has_no_secrets" {
+  count            = var.enabled ? 1 : 0
+  name             = var.config_rule_name
+  description      = "Ensures that Lambda function don't have secret keys in their environment variables"
+  input_parameters = "{\"ExecutionRoleName\": \"${var.config_rule_name}\"}"
+  source {
+    owner             = "CUSTOM_LAMBDA"
+    source_identifier = module.lambda_function.lambda_function_arn
+    source_detail {
+      event_source = "aws.config"
+      message_type = "ConfigurationItemChangeNotification"
+    }
+  }
+
+  depends_on = [
+    module.lambda_function
+  ]
+}

outputs.tf

@@ -0,0 +1,3 @@
+output "ConfigRuleName" {
+  value = var.config_rule_name == "" ? aws_config_config_rule.lambda_has_no_secrets[0].name : var.config_rule_name
+}

scripts/.DS_Store

@@ -0,0 +1,11 @@
+#!/bin/bash
+
+layers=( "rdklib" "detect-secrets")
+
+#Building layers
+for i in "${layers[@]}"
+do
+  ./build_layer.sh ${i}
+  zip -r ../src/layers/${i}-layer.zip ./python/
+  rm -rf python
+done

scripts/build.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+export PKG_DIR="python/lib/python3.8/site-packages"
+rm -rf ${PKG_DIR} && mkdir -p ${PKG_DIR}
+
+docker run --rm -v $(pwd):/package -w /package lambci/lambda:build-python3.8 \
+pip3 install $1 -t ${PKG_DIR}

scripts/build_layer.sh

@@ -0,0 +1,59 @@
+from rdklib import Evaluator, Evaluation, ConfigRule, ComplianceType
+from detect_secrets import SecretsCollection
+from detect_secrets.settings import default_settings
+import json
+
+APPLICABLE_RESOURCES = ['AWS::Lambda::Function']
+
+
+class lambda_has_no_secret_keys(ConfigRule):
+  def evaluate_change(self, event, client_factory, configuration_item,
+                      valid_rule_parameters):
+
+    lamda_client = client_factory.build_client('lambda')
+    resource_type = configuration_item.get('resourceType')
+
+    if resource_type == 'AWS::Lambda::Function':
+      resource_id = configuration_item.get('resourceId')
+
+      envVariables = get_env_variables(lamda_client, resource_id)
+
+      filename = "/tmp/variables.txt"
+      with open(filename, 'w+') as outfile:
+        json.dump(envVariables, outfile)
+
+      secrets = scan_file_for_secrets(filename)
+      if secrets:
+        return [Evaluation(ComplianceType.NON_COMPLIANT, resource_id,
+                           APPLICABLE_RESOURCES[0],
+                           json.dumps(secrets.json()[filename][0]["type"], indent=2))]
+      return [Evaluation(ComplianceType.COMPLIANT)]
+    return [Evaluation(ComplianceType.NOT_APPLICABLE)]
+
+  def evaluate_parameters(self, rule_parameters):
+    valid_rule_parameters = rule_parameters
+    return valid_rule_parameters
+
+
+def get_env_variables(lambda_client, resource_id):
+  lambda_config = lambda_client.get_function_configuration(
+    FunctionName=resource_id)
+  if lambda_config is not None:
+    return lambda_config.get('Environment').get('Variables')
+  return {}
+
+
+def scan_file_for_secrets(filename):
+  secrets = SecretsCollection()
+  with default_settings():
+    secrets.scan_file(filename)
+  return secrets
+
+
+################################
+# DO NOT MODIFY ANYTHING BELOW #
+################################
+def lambda_handler(event, context):
+  my_rule = lambda_has_no_secret_keys()
+  evaluator = Evaluator(my_rule, APPLICABLE_RESOURCES)
+  return evaluator.handle(event, context)