Merge pull request #21 from MicrosoftCloudEssentials-LearningHub/mongo-atlas

Mongo atlas -> Set Up a Network Peering Connection

Author: brown9804

1_storage-databases/README.md

@@ -5,7 +5,7 @@ Costa Rica
 [![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
 [brown9804](https://github.com/brown9804)
 
-Last updated: 2025-03-24
+Last updated: 2025-03-27
 
 ------------------------------------------
 
@@ -22,6 +22,7 @@ Last updated: 2025-03-24
 - [PostgreSQL Server](./1_storage-databases/postgresql)
 - [Azure SQL Database](./1_storage-databases/sql-database)
 - [Azure Storage Account](./1_storage-databases/storage-account)
+- [MongoDB Atlas with Azure Network Peering](./1_storage-databases/mongo-atlas)
 
 <div align="center">
   <h3 style="color: #4CAF50;">Total Visitors</h3>

1_storage-databases/mongo-atlas/README.md

@@ -0,0 +1,128 @@
+# Terraform Template - MongoDB Atlas with Azure Network Peering
+
+Costa Rica
+
+[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
+[brown9804](https://github.com/brown9804)
+
+Last updated: 2025-03-27
+
+----------
+
+> This template contains Terraform configurations to create and manage MongoDB Atlas with Azure network peering. Below is a description of the files and the variables used in this template.
+
+<details>
+<summary><b>List of References </b> (Click to expand)</summary>
+
+- [AzureRM - MongoDB Atlas Provider](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs)
+- [MongoDB - Set Up a Network Peering Connection](https://www.mongodb.com/docs/atlas/security-vpc-peering/)
+- [MongoDB - Manage Identity Providers](https://www.mongodb.com/docs/atlas/security/manage-federated-auth/)
+- [Resource: mongodbatlas_network_peering](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/network_peering)
+- [MongoDB Atlas - Limits](https://www.mongodb.com/docs/atlas/reference/limitations/#std-label-limits)
+- [Resource: azuread_service_principal](https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/resources/service_principal)
+- [Resource: azuread_application](https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/resources/application.html)
+- [Resource: azuread_service_principal_password](https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/resources/service_principal_password)
+- [Deploying MongoDB Atlas With Terraform with Azure](https://www.mongodb.com/developer/products/atlas/mongodb-atlas-deployment-with-terraform/?msockid=38ec3806873362243e122ce086486339)
+
+</details>
+
+
+> [!NOTE]
+> The MongoDB Atlas network peering depends on the Azure resources. Terraform ensures that these resources are created before the peering by using the `depends_on` argument in the configuration.
+
+<p align="center">
+    <img width="150" alt="image" src="https://github.com/user-attachments/assets/14f80abf-1cc2-4286-bb07-05dd24a0121d">
+</p>
+
+<p align="center">
+    <img width="550" alt="image" src="https://github.com/user-attachments/assets/2963fec6-0ec5-47b6-b203-a18b15a1d9a6">
+</p>
+
+<p align="center">
+    <img width="550" alt="image" src="https://github.com/user-attachments/assets/be7ee535-f2c8-4c33-9fa2-a75025366097">
+</p>
+
+<p align="center">
+    <img width="550" alt="image" src="https://github.com/user-attachments/assets/62f3c882-2ca5-4abf-9de0-fa83857ec03d">
+</p>
+
+
+<p align="center">
+    <img width="550" alt="image" src="https://github.com/user-attachments/assets/de1d3b05-1596-4be0-8a35-73d472b17478">
+</p>
+
+> [!TIP]
+> For this demostration, I used promotional codes that I found for MongoDB Atlas that will give you $100 FREE MONGODB credits: `GETATLAS `
+
+## Prerequisites
+
+- An `Azure subscription is required`. All other resources, including instructions for creating a Resource Group, are provided in this workshop.
+- `Contributor role assigned or any custom role that allows`: access to manage all resources, and the ability to deploy resources within subscription.
+- If you choose to use the Terraform approach, please ensure that:
+  -  [Terraform is installed on your local machine](https://developer.hashicorp.com/terraform/tutorials/azure-get-started/install-cli#install-terraform).
+  -  [Install the Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) to work with both Terraform and Azure commands.
+- A MongoDB Atlas account. If you don't have one yet, just sign up for a [free MongoDB Atlas account here](https://www.mongodb.com/cloud/atlas/register).
+
+    https://github.com/user-attachments/assets/1507a6a7-b74b-4ec6-8118-433ef52e6d8f
+
+## Permissions required 
+
+> Please read [terraform documentation on azurerm specifics](https://registry.terraform.io/providers/mongodb/mongodbatlas/latest/docs/resources/network_peering). To create the peering request with an Azure VNET, you need to grant Atlas the following permissions on the virtual network: <br/>
+> - Microsoft.Network/virtualNetworks/virtualNetworkPeerings/read <br/>
+> - Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write <br/>
+> - Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete <br/>
+> - Microsoft.Network/virtualNetworks/peer/action <br/>
+> For more information, see [Set Up a Network Peering Connection](https://docs.atlas.mongodb.com/security-vpc-peering/) and [Mongo DB network peering](https://docs.atlas.mongodb.com/reference/api/vpc-create-peering-connection/)
+
+> For example:
+
+https://github.com/user-attachments/assets/2f71a62f-1d8c-41f4-9403-efb84028056c
+
+## File Descriptions
+
+- **main.tf**: Contains the main configuration for creating the MongoDB Atlas network peering and the Azure resources that it depends on.
+- **variables.tf**: Defines the input variables used in the Terraform configuration.
+- **provider.tf**: Configures the providers to interact with MongoDB Atlas and Azure resources.
+- **terraform.tfvars**: Provides default values for the variables defined in `variables.tf`.
+- **outputs.tf**: Defines the outputs of the Terraform configuration, such as the peering connection details.
+
+## Variables
+
+Below is a list of variables used in this template, their expected values, types, and examples:
+
+| Variable Name               | Description                                      | Type    | Example Value             |
+|-----------------------------|--------------------------------------------------|---------|---------------------------|
+| `azure_subscription_id`     | The Azure subscription ID.                       | string  | `"your-subscription-id"`  |
+| `resource_group_name`       | The name of the Azure resource group.            | string  | `"your-resource-group"`   |
+| `vnet_name`                 | The name of the Azure virtual network.           | string  | `"your-vnet-name"`        |
+| `azure_vnet_cidr_block`     | The CIDR block for the Azure virtual network.    | string  | `"10.1.0.0/16"`           |
+| `atlas_cidr_block`          | The CIDR block for the MongoDB Atlas network container. | string  | `"192.168.1.0/24"`        |
+| `location`                  | The location/region where the resources will be created. | string  | `"West US"`               |
+| `mongodbatlas_org_id`       | The organization ID for MongoDB Atlas.           | string  | `"your-org-id"`           |
+| `azure_directory_id`        | The Azure directory ID.                          | string  | `"your-directory-id"`     |
+| `mongodbatlas_public_key`   | The public key for MongoDB Atlas.                | string  | `"your-public-key"`       |
+| `mongodbatlas_private_key`  | The private key for MongoDB Atlas.               | string  | `"your-private-key"`      |
+
+## How to create MongoDB Atlas public/private key
+
+1. Go to `Access Manager`
+2. Click on `Add new`
+
+   <img width="550" alt="image" src="https://github.com/user-attachments/assets/a96f3604-2e9c-4a72-8984-747577f2114d">
+
+3. Select API Key, click on create new, and store the values securely
+
+   <img width="350" alt="image" src="https://github.com/user-attachments/assets/4e774329-6af5-4e76-a6f8-13d25c30be08">
+        
+   https://github.com/user-attachments/assets/7c42f0c5-37cf-4d1b-9f48-ccf95d6c5f4d
+
+## How it works 
+
+> Overall, to set up network peering between Azure and MongoDB Atlas using Terraform, start by creating a MongoDB Atlas account, project, network container, and cluster. Ensure you have an Azure virtual network (VNet) configured and [grant necessary permissions to the Azure service principal to enable it to peer with MongoDB Atlas](#permissions-required). Deploy the VNet peering module to establish the connection between the Azure VNet and MongoDB Atlas, making sure the CIDR blocks do not overlap to avoid conflicts. You can use this template, and run the stardard terraform commands to deploy the infrastrcture, click [here if you need more information about it](https://github.com/MicrosoftCloudEssentials-LearningHub/AzureTerraformTemplates-v0.0.0/tree/mongo-atlas?tab=readme-ov-file#how-to-execute-it) <br/>
+
+https://github.com/user-attachments/assets/95f0c5cd-b1b9-422d-a32d-4cfdb98a5a1e
+
+<div align="center">
+  <h3 style="color: #4CAF50;">Total Visitors</h3>
+  <img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
+</div>

1_storage-databases/mongo-atlas/known_errors.md

@@ -0,0 +1,63 @@
+# Deployment known errors - Overview 
+
+Costa Rica
+
+[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
+[brown9804](https://github.com/brown9804)
+
+Last updated: 2025-03-27
+
+----------
+
+> Here is a brief overview of error messages and their solutions.
+
+## Content 
+
+- [401 - Unauthorized](#unauthorized)
+- [500 - Internal Server Error](#internal-server-error)
+
+## 401 Unauthorized 
+
+<p align="center">
+  <img width="550" alt="image" src="https://github.com/user-attachments/assets/7c42ef92-6f1e-4dac-a105-f0b5ed4dee4c" />
+</p>
+
+```
+│ Error: error creating project: Terraform Project
+│
+│   with mongodbatlas_project.example,
+│   on main.tf line 42, in resource "mongodbatlas_project" "example":
+│   42: resource "mongodbatlas_project" "example" {
+│
+│ https://cloud.mongodb.com/api/atlas/v2/groups POST: HTTP 401 Unauthorized (Error code: "") Detail: You are not     
+│ authorized for this resource. Reason: Unauthorized. Params: [], BadRequestDetail:
+```
+
+> [!TIP]
+> Ensure you have added the user, group, or identity to the Atlas organization. This issue can also be resolved by leveraging public and private keys to connect. Check [this for more information about it](
+
+## 500 Internal Server Error
+
+<p align="center">
+  <img width="550" alt="image" src="https://github.com/user-attachments/assets/482a289c-e912-40e2-b13a-4167a763cdfe" />
+</p>
+
+
+```
+╷
+│ Error: error creating MongoDB Network Peering Connection: https://cloud.mongodb.com/api/atlas/v2/groups/677a5/peers POST: HTTP 500 Internal Server Error (Error code: "UNEXPECTED_ERROR") Detail: Unexpected error. Reason: Internal Server Error. Params: [], BadRequestDetail: 
+│
+│   with mongodbatlas_network_peering.azure_mongodb,
+│   on main.tf line 71, in resource "mongodbatlas_network_peering" "azure_mongodb":
+│   71: resource "mongodbatlas_network_peering" "azure_mongodb" {
+│
+```
+
+> [!TIP]
+> Please ensure you have configured the Terraform module correctly. You can use [this template as a guide](/1_storage-databases/mongo-atlas/main.tf).
+> Also, make sure you have created the service principal with the correct permissions. [Validate the connection before continuing, as shown in the video](https://github.com/MicrosoftCloudEssentials-LearningHub/AzureTerraformTemplates-v0.0.0/tree/main/1_storage-databases/mongo-atlas#permissions-required).
+ 
+<div align="center">
+  <h3 style="color: #4CAF50;">Total Visitors</h3>
+  <img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
+</div>

1_storage-databases/mongo-atlas/main.tf

@@ -0,0 +1,111 @@
+# Main configuration file for defining MongoDB Atlas network peering and Azure resources
+
+# Notes:
+# To assign the necessary permissions to the user running Terraform, please review
+# how to create a service principal (if you don't have one) and assign a custom role to it.
+# - Use the `azurerm_resource_group` resource to create a resource group in Azure. 
+#   This is a container that holds related resources for an Azure solution.
+# - Use the `azurerm_virtual_network` resource to create a virtual network in Azure. 
+#   This network will be used for the peering connection with MongoDB Atlas.
+# - Use the `mongodbatlas_project` resource to create a project in MongoDB Atlas. 
+#   A project is a logical grouping of clusters, users, and other resources.
+# - Use the `mongodbatlas_network_container` resource to define the CIDR block 
+#   and provider details for the peering connection. This container holds the 
+#   network settings for the peering connection.
+# - Use the `mongodbatlas_network_peering` resource to create 
+#   the peering connection request with Azure VNET. This resource 
+#   establishes the network peering between MongoDB Atlas and Azure.
+# - The `mongodbatlas_advanced_cluster` resource is used to create a MongoDB Atlas cluster once the peering connection is established. 
+#   This resource defines the cluster configuration, including replication specs 
+#   and backup settings. Note that VNET peering is only supported for dedicated clusters (M10 and above).
+
+#----------------------------------------------------------------
+# Create the Azure Resource Group
+#----------------------------------------------------------------
+resource "azurerm_resource_group" "example" {
+  name     = var.resource_group_name
+  location = var.location
+}
+
+#----------------------------------------------------------------
+# Create the Azure Virtual Network
+#----------------------------------------------------------------
+resource "azurerm_virtual_network" "example" {
+  name                = var.vnet_name
+  address_space       = [var.azure_vnet_cidr_block] # Ensure this CIDR block does not overlap with existing peering connections
+  location            = var.location
+  resource_group_name = azurerm_resource_group.example.name
+}
+
+#----------------------------------------------------------------
+# Create the MongoDB Atlas Project
+#----------------------------------------------------------------
+resource "mongodbatlas_project" "example" {
+  name   = "Terraform Project" # Change for your project name 
+  org_id = var.mongodbatlas_org_id
+}
+
+#----------------------------------------------------------------
+# Create the MongoDB Atlas Network Container
+# Container example provided but not always required,
+# see network_container documentation for details.
+#----------------------------------------------------------------
+resource "mongodbatlas_network_container" "example" {
+  project_id       = mongodbatlas_project.example.id
+  atlas_cidr_block = var.atlas_cidr_block
+  provider_name    = "AZURE"
+  region           = "US_EAST_2"
+}
+
+#----------------------------------------------------------------
+# Create the MongoDB Atlas Cluster
+#----------------------------------------------------------------
+resource "mongodbatlas_advanced_cluster" "test" {
+  project_id     = mongodbatlas_project.example.id
+  name           = "terraform-deployment-test" # Choose cluster name 
+  cluster_type   = "REPLICASET"
+  backup_enabled = true
+  replication_specs {
+    region_configs {
+      priority      = 7
+      provider_name = "AZURE"
+      region_name   = "US_EAST_2"
+      electable_specs {
+        instance_size = "M10" # Depending on the tier, clusters have certain configuration limits
+        node_count    = 3
+      }
+    }
+  }
+
+  depends_on = [mongodbatlas_network_container.example]
+}
+
+#----------------------------------------------------------------
+# Introduce a wait time before creating the MongoDB Atlas Network Peering
+#----------------------------------------------------------------
+resource "time_sleep" "wait_5_minutes" {
+  depends_on = [
+    mongodbatlas_advanced_cluster.test
+  ]
+
+  create_duration = "5m"
+}
+
+#----------------------------------------------------------------
+# Create the MongoDB Atlas Network Peering
+#----------------------------------------------------------------
+resource "mongodbatlas_network_peering" "azure_mongodb" {
+  project_id            = mongodbatlas_project.example.id
+  container_id          = mongodbatlas_network_container.example.id
+  provider_name         = "AZURE"
+  azure_directory_id    = var.azure_directory_id
+  azure_subscription_id = var.azure_subscription_id
+  resource_group_name   = var.resource_group_name
+  vnet_name             = var.vnet_name
+
+  depends_on = [
+    azurerm_resource_group.example,
+    azurerm_virtual_network.example,
+    time_sleep.wait_5_minutes
+  ]
+}

1_storage-databases/mongo-atlas/outputs.tf

@@ -0,0 +1,29 @@
+output "resource_group_name" {
+  description = "The name of the resource group created in Azure."
+  value       = azurerm_resource_group.example.name
+}
+
+output "vnet_name" {
+  description = "The name of the virtual network created in Azure."
+  value       = azurerm_virtual_network.example.name
+}
+
+output "mongodbatlas_project_id" {
+  description = "The ID of the MongoDB Atlas project."
+  value       = mongodbatlas_project.example.id
+}
+
+output "mongodbatlas_network_container_id" {
+  description = "The ID of the MongoDB Atlas network container."
+  value       = mongodbatlas_network_container.example.id
+}
+
+output "mongodbatlas_network_peering_id" {
+  description = "The ID of the MongoDB Atlas network peering."
+  value       = mongodbatlas_network_peering.azure_mongodb.id
+}
+
+output "mongodbatlas_cluster_id" {
+  description = "The ID of the MongoDB Atlas cluster."
+  value       = mongodbatlas_advanced_cluster.test.id
+}

1_storage-databases/mongo-atlas/peering-role.json

@@ -0,0 +1,14 @@
+{
+    "Name": "AtlasPeering",
+    "IsCustom": true,
+    "Description": "Grants MongoDB access to manage peering connections on network",
+    "Actions": [
+        "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/read",
+        "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write",
+        "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete",
+        "Microsoft.Network/virtualNetworks/peer/action"
+    ],
+    "AssignableScopes": [
+        "/subscriptions/{your-subscription-id}"
+    ]
+}