Skip to content

Commit 8e2fa30

Browse files
author
Yoichi Kawasaki
committed
Initial commit
1 parent 1175cf6 commit 8e2fa30

File tree

15 files changed

+784
-1
lines changed

15 files changed

+784
-1
lines changed

‎README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
1-
# azure-db-scaler
1+
# azure-database-scaler
22
Event driven Logic App and Functions that scale-up or scale-down the capacity of your Azure database service instance (Currently only vCore # is supported)
3+
4+
5+
## How to deploy the scaler app
6+
7+
8+
## How to configure Azure Database autoscale using Azure Alerts

‎docs/HOW-TO-DEPLOY-APP.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# How to deploy the demo application
2+
3+
## Prerequisites
4+
You can run this walkthrough on Linux (of course, Azure Cloud Shell Bash) or Mac OS. Maybe Bash on Windows too but not tested yet
5+
6+
- You need an Azure subscription. If you don't have one, you can [sign up for an account](https://azure.microsoft.com/).
7+
- Install the [Azure CLI 2.0](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)
8+
```
9+
pip install -U azure-cli
10+
```
11+
[NOTE] you can skip azure-cli installation if you execute bash commands from [Azure Cloud Shell](https://docs.microsoft.com/en-us/azure/cloud-shell/overview). I recommend you to use [Azure Cloud Shell Bash](https://docs.microsoft.com/en-us/azure/cloud-shell/overview) as it's browser based and you can run anywhere
12+
13+
## Create Resources and Deploy applications
14+
15+
### 1. Create Resource Group
16+
Create resource group for the demo (ie.,Resource group named `rg-dbscalerapp` in `japanwest` region):
17+
```
18+
az group create --name rg-dbscalerapp --location japanwest
19+
```
20+
21+
### 2. Service principal for the app
22+
23+
Create a Service Principal with the following Azure CLI command:
24+
```
25+
az ad sp create-for-rbac --role Contributor
26+
```
27+
Output should be similar to the following. Take note of the appId, password, and tenant values, which you use in the next step.
28+
29+
```
30+
{
31+
"appId": "333bfe4e-e98c-4d78-adb1-947d45296caa", # to clientId in project.conf
32+
"displayName": "azure-cli-2018-04-28-02-12-13",
33+
"name": "http://azure-cli-2018-04-28-02-12-13",
34+
"password": "3d3f4303-ca12-4c29-94a7-e5bde0c1f8c1", # to clientSecret in project.conf
35+
"tenant": "72f988bf-86f1-41af-91ab-2d7cd011db47" # to tenantId in project.conf
36+
}
37+
```
38+
39+
### 3. Clone the project source code from Github and edit project.conf
40+
41+
Clone the project source code from Github
42+
```
43+
git clone https://github.com/yokawasa/azure-database-scaler.git
44+
```
45+
46+
Then, open `project.conf` and add all except `WebhookSubscribeAPIEndpoint` parameters in project.conf.
47+
- azure-database-scaler/scripts/[project.conf](../scripts/project.conf)
48+
```
49+
#=============================================================
50+
# Azure Database Scaler App Configuration file
51+
#=============================================================
52+
# Commons
53+
ResourceGroup="<Resource group for the project>"
54+
ResourceLocation="<Resource Region name (ex. japaneast)>"
55+
SubscriptionId="<Azure Subscription ID>"
56+
TenantId="<Service Principal Tenant Domain>"
57+
ClientId="<Service Principal Client Id>"
58+
ClientSecret="<Service Principal Client Secret>"
59+
60+
# Functions App
61+
FunctionsAppName="<Function App Name>"
62+
FunctionsAppConsumptionPlanLocation="<Location/Region name for running function apps>"
63+
WebhookSubscribeAPIEndpoint="https://${FunctionsAppName}.azurewebsites.net/api/webhookhandler?code=xxx"
64+
65+
# Logic App
66+
LogicAppName="<Logic App Name>"
67+
68+
# Storage Account for the App
69+
StorageAccountName="<Storage Account Name>"
70+
71+
# Database Admin/Login User in Azure Database Services (MySQL/PostgreSQL)
72+
DatabaseAdminUser="<Database Admin/Login User Name in your Azure Database service>"
73+
DatabaseAdminPassword="<Database Admin User Password in your Azure Database service>"
74+
```
75+
[NOTE] Your Functions App, LogicApp, and Storage Account names must be unique within Azure.
76+
77+
### 4. Create Azure Steorage Account
78+
79+
Create Azure Storage Account for the app by running a following scirpt:
80+
```
81+
scripts/setup-storage.sh
82+
```
83+
84+
### 5. Create Azure functions account and deploy functions app
85+
Create Azure Functions Account and deploy functions apps into the account by running a following scirpt:
86+
```
87+
scripts/setup-functions.sh
88+
```
89+
90+
### 6. Get webhookhandler function URL and update project.conf
91+
92+
Get the following functions' URLs (scheme + host/path + query) in the Azure Portal:
93+
- webhookhandler
94+
95+
Here is how you get get-sas-token function's URL.
96+
![](../images/screenshot-functions-url.png)
97+
98+
Once you get `webhookhandler` function URL, replace `WebhookSubscribeAPIEndpoint` value with `webhookhandler` function URL in `project.conf` like this:
99+
```
100+
WebhookSubscribeAPIEndpoint="https://dbscalerapp.azurewebsites.net/api/webhookhandler?code=lylgAi/RUgh/vZxby34/lsKlOBDa4HR1teNrV5qHUbuoMRrEAlI36w=="
101+
```
102+
103+
### 7. Deploy Azure Logic App
104+
105+
Create Azure Logic App Account and deploy a workflow to the account by running a following scirpt:
106+
```
107+
scripts/setup-logicapp.sh
108+
```
109+
110+
### 8. Configure Connection to your Slack account
111+

‎functions/dbscaler/function.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"bindings": [
3+
{
4+
"name": "myQueueItem",
5+
"type": "queueTrigger",
6+
"direction": "in",
7+
"queueName": "scalerqueue",
8+
"connection": "STORAGE_CONNECTION"
9+
}
10+
],
11+
"disabled": false
12+
}

‎functions/dbscaler/project.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"frameworks": {
3+
"net46": {
4+
"dependencies": {
5+
"Microsoft.Azure.Management.ResourceManager":"1.6.0-preview",
6+
"Microsoft.IdentityModel.Clients.ActiveDirectory":"2.28.3",
7+
"Microsoft.Rest.ClientRuntime": "2.3.9",
8+
"Microsoft.Rest.ClientRuntime.Azure":"3.3.5",
9+
"Microsoft.Rest.ClientRuntime.Azure.Authentication":"2.3.2"
10+
}
11+
}
12+
}
13+
}
14+

‎functions/dbscaler/run.csx‎

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
#r "Newtonsoft.Json"
2+
#r "System.Runtime.Serialization"
3+
4+
//*************************************************************
5+
// Azure Database for MySQL/PostgreSQL Scaling Down/Up Functions
6+
//
7+
// For REST API params, refer to
8+
// https://docs.microsoft.com/en-us/rest/api/mysql/servers/update
9+
//**************************************************************
10+
11+
using System;
12+
using Newtonsoft.Json;
13+
using Newtonsoft.Json.Linq;
14+
using System.Net.Http;
15+
using Microsoft.Rest.Azure.Authentication;
16+
using Microsoft.Azure.Management.ResourceManager;
17+
using Microsoft.Azure.Management.ResourceManager.Models;
18+
19+
//private const string _MySQL_Namespace = "Microsoft.DBforMySQL";
20+
//private const string _PostgreSQL_Namespace = "Microsoft.DBforPostgreSQL";
21+
private const string _apiVersion = "2017-12-01";
22+
private const string _dbTier_basic = "Basic";
23+
private const string _dbTier_generalPurpose = "GeneralPurpose";
24+
private const string _dbTier_memoryOptimized = "MemoryOptimized";
25+
private static int[] _scaleModel_basic = {1,2};
26+
private static int[] _scaleModel_generalPurpose = { 2, 4, 8, 16, 32 };
27+
private static int[] _scaleModel_memoryOptimized = { 2, 4, 8, 16, 32 };
28+
29+
private static readonly string _tenantId = Environment.GetEnvironmentVariable("AZURE_TENANT_ID");
30+
private static readonly string _clientId = Environment.GetEnvironmentVariable("AZURE_CLIENT_ID");
31+
private static readonly string _clientSecret = Environment.GetEnvironmentVariable("AZURE_CLIENT_SECRET");
32+
private static readonly string _subscriptionId = Environment.GetEnvironmentVariable("AZURE_SUBSCRIPTION_ID");
33+
private static readonly string _dbAdminUser = Environment.GetEnvironmentVariable("DB_ADMIN_USER");
34+
private static readonly string _dbAdminPassword = Environment.GetEnvironmentVariable("DB_ADMIN_PASSWORD");
35+
36+
private static HttpClient client = new HttpClient();
37+
38+
public static void Run(string myQueueItem, TraceWriter log)
39+
{
40+
log.Info($"queueTrigger Function was triggerd");
41+
string jsonContent = myQueueItem;
42+
dynamic data = JsonConvert.DeserializeObject(jsonContent);
43+
log.Info("Request : " + jsonContent);
44+
string callbackUrl = data.CallbackUrl;
45+
46+
dynamic context = JsonConvert.DeserializeObject((string)data.Context);
47+
string resourceId = (string) context["resourceId"];
48+
string resourceRegion = (string) context["resourceRegion"];
49+
dynamic condition = context["condition"].ToObject<Dictionary<string, string>>();
50+
string alertOperator = condition["operator"];
51+
log.Info("Input - callbackUrl : " + callbackUrl);
52+
log.Info("Input - resourceId : " + resourceId);
53+
log.Info("Input - resourceRegion : " + resourceRegion);
54+
log.Info("Input - alertOperator : " + alertOperator);
55+
56+
string[] restokens = resourceId.Split('/');
57+
if (restokens.Length != 9 ) {
58+
var errmsg=string.Format("Invalid resource Id: {0}", resourceId);
59+
log.Info(errmsg);
60+
throw new Exception(errmsg);
61+
}
62+
63+
string dbResourceGroup = restokens[4];
64+
string resourceProviderNamespace=restokens[6];
65+
string dbName = restokens[8];
66+
log.Info(string.Format("Input - dbResourceGroup:{0} resourceProviderNamespace:{1} dbName:{2}",
67+
dbResourceGroup,
68+
resourceProviderNamespace,
69+
dbName ));
70+
71+
if (new List<string> {
72+
_tenantId,
73+
_clientId,
74+
_clientSecret,
75+
_subscriptionId,
76+
_dbAdminUser,
77+
_dbAdminPassword
78+
}.Any(i => String.IsNullOrEmpty(i)))
79+
{
80+
string errmsg="[ERROR] Please provide ENV vars for AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_SECRET, AZURE_SUBSCRIPTION_ID, DB_ADMIN_USER, and DB_ADMIN_PASSWORD";
81+
Console.WriteLine(errmsg);
82+
throw new Exception(errmsg);
83+
}
84+
85+
try {
86+
// Db vCore Scale Change
87+
ScaleDatabaeVCoreCapacity(
88+
resourceRegion,
89+
dbResourceGroup,
90+
resourceProviderNamespace,
91+
dbName,
92+
alertOperator,
93+
log
94+
).Wait();
95+
}
96+
catch (Exception ex)
97+
{
98+
log.Info("[Exception] " + ex);
99+
throw new Exception("[Exception] " + ex);
100+
}
101+
102+
string result = jsonContent;
103+
client.PostAsJsonAsync<string>(callbackUrl, result);
104+
}
105+
106+
107+
public static async Task ScaleDatabaeVCoreCapacity(
108+
string region,
109+
string resourceGroupName,
110+
string resourceProviderNamespace,
111+
string resourceName,
112+
string alertOperator,
113+
TraceWriter log
114+
)
115+
{
116+
117+
// Build the service credentials and Azure Resource Manager clients
118+
var serviceCreds =
119+
await ApplicationTokenProvider.LoginSilentAsync(_tenantId, _clientId, _clientSecret);
120+
var resourceClient =
121+
new ResourceManagementClient(serviceCreds);
122+
resourceClient.SubscriptionId = _subscriptionId;
123+
124+
//***************************************
125+
// Get current SKU & Properties
126+
//***************************************
127+
GenericResource gr;
128+
gr = resourceClient.Resources.Get(
129+
resourceGroupName,
130+
resourceProviderNamespace, // Microsoft.DBforMySQL or Microsoft.DBforPostgreSQL
131+
"", // fixed
132+
"servers", // fixed "servers"
133+
resourceName, // db account name
134+
_apiVersion // API version fixed
135+
);
136+
Sku cursku = gr.Sku;
137+
log.Info( string.Format("Current SKU: name:{0}, Tier:{1}, Family:{2}, Capacity:{3}",
138+
cursku.Name, cursku.Tier, cursku.Family, cursku.Capacity));
139+
140+
JObject curProperties_jobject = (JObject)gr.Properties;
141+
Dictionary<string, object> newProperties = curProperties_jobject.ToObject<Dictionary<string, object>>();
142+
Sku newsku = cursku;
143+
144+
//***************************************
145+
// Decide Capacity
146+
//***************************************
147+
int[] scaleModel;
148+
string tierAcronym;
149+
switch (cursku.Tier)
150+
{
151+
case _dbTier_basic:
152+
log.Info("Basic Tier");
153+
scaleModel = _scaleModel_basic;
154+
tierAcronym = "B";
155+
break;
156+
case _dbTier_generalPurpose:
157+
log.Info("GeneralPurpose Tier");
158+
scaleModel = _scaleModel_generalPurpose;
159+
tierAcronym = "GP";
160+
break;
161+
case _dbTier_memoryOptimized:
162+
log.Info("MemoryOptimized Tier");
163+
scaleModel = _scaleModel_memoryOptimized;
164+
tierAcronym = "MO";
165+
break;
166+
default:
167+
log.Info("[Warning] Unknown Tier, then skip the rest of operation");
168+
return;
169+
}
170+
171+
int direction = 0;
172+
if ( alertOperator.Equals("GreaterThan") || alertOperator.Equals("GreaterThanOrEqual"))
173+
{
174+
direction = 1;
175+
} else if (alertOperator.Equals("LessThan") || alertOperator.Equals("LessThanOrEqual") )
176+
{
177+
direction = -1;
178+
}
179+
int curIndex = 0;
180+
int newCapacity = (int)cursku.Capacity; // by default
181+
foreach (int c in scaleModel)
182+
{
183+
if (c == (int)cursku.Capacity)
184+
{
185+
if (direction > 0 && (scaleModel.Length - 1) > curIndex)
186+
{
187+
newCapacity = scaleModel[curIndex + 1];
188+
break;
189+
}
190+
if (direction < 0 && 0 < curIndex)
191+
{
192+
newCapacity = scaleModel[curIndex - 1];
193+
break;
194+
}
195+
}
196+
curIndex++;
197+
}
198+
199+
//***************************************
200+
// Update Sku
201+
//***************************************
202+
if (newCapacity == (int)cursku.Capacity)
203+
{
204+
log.Info(
205+
string.Format(
206+
"Skip Database vCore Update as it has reached max or min number: Name: {0}",
207+
resourceName) );
208+
return;
209+
}
210+
newsku.Capacity = newCapacity;
211+
newsku.Name = string.Format("{0}_{1}_{2}", tierAcronym, newsku.Family, newCapacity);
212+
log.Info(
213+
string.Format("New SKU: name:{0}, Tier:{1}, Family:{2}, Capacity:{3}",
214+
newsku.Name, newsku.Tier, newsku.Family, newsku.Capacity));
215+
216+
// AdminLoginPassword need to be added anytime of updating db resource
217+
// since it isn't loaded in properties object
218+
if (!newProperties.ContainsKey("administratorLoginPassword"))
219+
{
220+
newProperties.Add("administratorLoginPassword", _dbAdminPassword);
221+
newProperties["administratorLogin"] = _dbAdminUser; //Set dbAdminuser together with the password for its consistency
222+
}
223+
var databaseParams = new GenericResource
224+
{
225+
Location = region,
226+
Sku = newsku,
227+
Properties = newProperties
228+
};
229+
var database = resourceClient.Resources.CreateOrUpdate(
230+
resourceGroupName,
231+
resourceProviderNamespace, // Microsoft.DBforMySQL or Microsoft.DBforPostgreSQL
232+
"", // fixed
233+
"servers", // fixed
234+
resourceName, // db account name
235+
_apiVersion, // API Version fixed
236+
databaseParams ); // parameters
237+
238+
log.Info(
239+
string.Format("Database vCore scaling completed successfully: Name: {0} and Id: {1}",
240+
database.Name, database.Id));
241+
242+
}

0 commit comments

Comments
 (0)