kvm: Aqcuire lock when running security group Python script

It could happen that when multiple instances are starting at the same
time on a KVM host the Agent spawns multiple instances of security_group.py
which both try to modify iptables/ebtables rules.

This fails with on of the two processes failing.

The instance is still started, but it doesn't have any IP connectivity due
to the failed programming of the security groups.

This modification lets the script aqcuire a exclusive lock on a file so that
only one instance of the scripts talks to iptables/ebtables at once.

Other instances of the script which start will poll every 100ms if they can
obtain the lock and otherwise execute anyway after 10 seconds.

Author: wido

scripts/vm/network/security_group.py

@@ -26,8 +26,11 @@
 from optparse import OptionParser, OptionGroup, OptParseError, BadOptionError, OptionError, OptionConflictError, OptionValueError
 import re
 import libvirt
+import fcntl
+import time
 
 logpath = "/var/run/cloud/"        # FIXME: Logs should reside in /var/log/cloud
+lock_file = "/var/lock/cloudstack_security_group.lock"
 iptables = Command("iptables")
 bash = Command("/bin/bash")
 ebtables = Command("ebtables")
@@ -36,6 +39,18 @@
 hyper = cfo.getEntry("hypervisor.type")
 if hyper == "lxc":
     driver = "lxc:///"
+
+lock_handle = None
+
+def file_is_locked(path):
+    global lock_handle
+    lock_handle = open(path, 'w')
+    try:
+        fcntl.lockf(lock_handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        return False
+    except IOError:
+        return True
+
 def execute(cmd):
     logging.debug(cmd)
     return bash("-c", cmd).stdout
@@ -1029,6 +1044,13 @@ def addFWFramework(brname):
         sys.exit(1)
     cmd = args[0]
     logging.debug("Executing command: " + str(cmd))
+
+    for i in range(0,100):
+        if file_is_locked(lock_file):
+            time.sleep(0.1)
+        else:
+            break
+
     if cmd == "can_bridge_firewall":
         can_bridge_firewall(args[1])
     elif cmd == "default_network_rules":