Editing README

Author: Mohamed Ameen

README.md

@@ -1,7 +1,7 @@
 # **Lane Lines Detection Using Python and OpenCV** 
-In this project, I used Python and OpenCV to detect lane lines on the road. # I developed a processing pipeline that works on a series of individual images, and applied the result to a video stream.
+In this project, I used Python and OpenCV to detect lane lines on the road. I developed a processing pipeline that works on a series of individual images, and applied the result to a video stream.
 
-<img src="examples/laneLines_thirdPass.jpg" width="480" alt="Combined Image" />
+![jpg](examples/laneLines_thirdPass.jpg)
 
 Pipeline architecture:
 ---
@@ -17,10 +17,249 @@ Pipeline architecture:
 7. Apply on video streams.
 I'll explain each step in details below.
 
-Environement:
+## **Environement:**
 ---
 - Windows 7
 - Anaconda 4.3.29
 - Python 3.6.2
 - OpenCV 3.1.0
 
+### **1. Loading test images:**
+----
+We have 6 test images. I wrote a function called `list_images()` that shows all the test images we're working on using matplotlib.
+
+```python
+def list_images(images, cols = 2, rows = 5, cmap=None):
+    """
+    Display a list of images in a single figure with matplotlib.
+        Parameters:
+            images: List of np.arrays compatible with plt.imshow.
+            cols (Default = 2): Number of columns in the figure.
+            rows (Default = 5): Number of rows in the figure.
+            cmap (Default = None): Used to display gray images.
+    """
+    plt.figure(figsize=(10, 11))
+    for i, image in enumerate(images):
+        plt.subplot(rows, cols, i+1)
+        #Use gray scale color map if there is only one channel
+        cmap = 'gray' if len(image.shape) == 2 else cmap
+        plt.imshow(image, cmap = cmap)
+        plt.xticks([])
+        plt.yticks([])
+    plt.tight_layout(pad=0, h_pad=0, w_pad=0)
+    plt.show()
+```
+
+### **2. Color Selection:**
+----
+Lane lines in the test images are in white and yellow. We need to choose the most suitable color space, that clearly highlights the lane lines.
+I applied color selection to the original RGB images, HSV images, and HSL images, and found out that using HSL will be the best color space to use.
+
+```python
+def HSL_color_selection(image):
+    """
+    Apply color selection to the HSL images to blackout everything except for white and yellow lane lines.
+        Parameters:
+            image: An np.array compatible with plt.imshow.
+    """
+    #Convert the input image to HSL
+    converted_image = convert_hsl(image)
+    
+    #White color mask
+    lower_threshold = np.uint8([0, 200, 0])
+    upper_threshold = np.uint8([255, 255, 255])
+    white_mask = cv2.inRange(converted_image, lower_threshold, upper_threshold)
+    
+    #Yellow color mask
+    lower_threshold = np.uint8([10, 0, 100])
+    upper_threshold = np.uint8([40, 255, 255])
+    yellow_mask = cv2.inRange(converted_image, lower_threshold, upper_threshold)
+    
+    #Combine white and yellow masks
+    mask = cv2.bitwise_or(white_mask, yellow_mask)
+    masked_image = cv2.bitwise_and(image, image, mask = mask)
+    
+    return masked_image
+```
+
+### **3. Canny Edge Detection**
+----
+We need to detect edges in the images to be able to correctly detect lane lines.
+The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images.
+The Canny algorithm involves the following steps:
+- Gray scaling the images: The Canny edge detection algorithm measures the intensity gradients of each pixel. So, we need to convert the images into gray scale in order to detect edges.
+- Gaussian smoothing: Since all edge detection results are easily affected by image noise, it is essential to filter out the noise to prevent false detection caused by noise. To smooth the image, a Gaussian filter is applied to convolve with the image. This step will slightly smooth the image to reduce the effects of obvious noise on the edge detector.
+- Find the intensity gradients of the image.
+- Apply non-maximum suppression to get rid of spurious response to edge detection.
+- Apply double threshold to determine potential edges.
+- Track edge by hysteresis: Finalize the detection of edges by suppressing all the other edges that are weak and not connected to strong edges.
+If an edge pixel’s gradient value is higher than the high threshold value, it is marked as a strong edge pixel. If an edge pixel’s gradient value is smaller than the high threshold value and larger than the low threshold value, it is marked as a weak edge pixel. If an edge pixel's value is smaller than the low threshold value, it will be suppressed. The two threshold values are empirically determined and their definition will depend on the content of a given input image.
+
+### **4. Region of interest**
+----
+We're interested in the area facing the camera, where the lane lines are found. So, we'll apply region masking to cut out everything else.
+
+```python
+def region_selection(image):
+    """
+    Determine and cut the region of interest in the input image.
+        Parameters:
+            image: An np.array compatible with plt.imshow.
+    """
+    mask = np.zeros_like(image)   
+    #Defining a 3 channel or 1 channel color to fill the mask with depending on the input image
+    if len(image.shape) > 2:
+        channel_count = image.shape[2]
+        ignore_mask_color = (255,) * channel_count
+    else:
+        ignore_mask_color = 255
+    #We could have used fixed numbers as the vertices of the polygon,
+    #but they will not be applicable to images with different dimesnions.
+    rows, cols = image.shape[:2]
+    bottom_left  = [cols * 0.1, rows * 0.95]
+    top_left     = [cols * 0.4, rows * 0.6]
+    bottom_right = [cols * 0.9, rows * 0.95]
+    top_right    = [cols * 0.6, rows * 0.6]
+    vertices = np.array([[bottom_left, top_left, top_right, bottom_right]], dtype=np.int32)
+    cv2.fillPoly(mask, vertices, ignore_mask_color)
+    masked_image = cv2.bitwise_and(image, mask)
+    return masked_image
+```
+
+### **5. Hough Transform**
+----
+The Hough transform is a technique which can be used to isolate features of a particular shape within an image. I'll use it to detected the lane lines in `selected_region_images`.
+
+```python
+def hough_transform(image):
+    """
+    Determine and cut the region of interest in the input image.
+        Parameters:
+            image: The output of a Canny transform.
+    """
+    rho = 1              #Distance resolution of the accumulator in pixels.
+    theta = np.pi/180    #Angle resolution of the accumulator in radians.
+    threshold = 20       #Only lines that are greater than threshold will be returned.
+    minLineLength = 20   #Line segments shorter than that are rejected.
+    maxLineGap = 300     #Maximum allowed gap between points on the same line to link them
+    return cv2.HoughLinesP(image, rho = rho, theta = theta, threshold = threshold,
+                           minLineLength = minLineLength, maxLineGap = maxLineGap)
+```
+
+### **6. Averaging and extrapolating the lane lines**
+----
+We have multiple lines detected for each lane line. We need to average all these lines and draw a single line for each lane line. We also need to extrapolate the lane lines to cover the full lane line length.
+```python
+def average_slope_intercept(lines):
+    """
+    Find the slope and intercept of the left and right lanes of each image.
+        Parameters:
+            lines: The output lines from Hough Transform.
+    """
+    left_lines    = [] #(slope, intercept)
+    left_weights  = [] #(length,)
+    right_lines   = [] #(slope, intercept)
+    right_weights = [] #(length,)
+    
+    for line in lines:
+        for x1, y1, x2, y2 in line:
+            if x1 == x2:
+                continue
+            slope = (y2 - y1) / (x2 - x1)
+            intercept = y1 - (slope * x1)
+            length = np.sqrt(((y2 - y1) ** 2) + ((x2 - x1) ** 2))
+            if slope < 0:
+                left_lines.append((slope, intercept))
+                left_weights.append((length))
+            else:
+                right_lines.append((slope, intercept))
+                right_weights.append((length))
+    left_lane  = np.dot(left_weights,  left_lines) / np.sum(left_weights)  if len(left_weights) > 0 else None
+    right_lane = np.dot(right_weights, right_lines) / np.sum(right_weights) if len(right_weights) > 0 else None
+    return left_lane, right_lane
+
+def pixel_points(y1, y2, line):
+    """
+    Converts the slope and intercept of each line into pixel points.
+        Parameters:
+            y1: y-value of the line's starting point.
+            y2: y-value of the line's end point.
+            line: The slope and intercept of the line.
+    """
+    if line is None:
+        return None
+    slope, intercept = line
+    x1 = int((y1 - intercept)/slope)
+    x2 = int((y2 - intercept)/slope)
+    y1 = int(y1)
+    y2 = int(y2)
+    return ((x1, y1), (x2, y2))
+
+def lane_lines(image, lines):
+    """
+    Create full lenght lines from pixel points.
+        Parameters:
+            image: The input test image.
+            lines: The output lines from Hough Transform.
+    """
+    left_lane, right_lane = average_slope_intercept(lines)
+    y1 = image.shape[0]
+    y2 = y1 * 0.6
+    left_line  = pixel_points(y1, y2, left_lane)
+    right_line = pixel_points(y1, y2, right_lane)
+    return left_line, right_line
+    
+def draw_lane_lines(image, lines, color=[255, 0, 0], thickness=12):
+    """
+    Draw lines onto the input image.
+        Parameters:
+            image: The input test image.
+            lines: The output lines from Hough Transform.
+            color (Default = red): Line color.
+            thickness (Default = 12): Line thickness. 
+    """
+    line_image = np.zeros_like(image)
+    for line in lines:
+        if line is not None:
+            cv2.line(line_image, *line,  color, thickness)
+    return cv2.addWeighted(image, 1.0, line_image, 1.0, 0.0)
+```
+
+
+### **7. Apply on video streams**
+----
+Now, we'll use the above functions to detect lane lines from a video stream.
+The video inputs are in test_videos folder. The video outputs are generated in output_videos folder.
+
+```python
+def frame_processor(image):
+    """
+    Process the input frame to detect lane lines.
+        Parameters:
+            image: Single video frame.
+    """
+    color_select = HSL_color_selection(image)
+    gray         = gray_scale(color_select)
+    smooth       = gaussian_smoothing(gray)
+    edges        = canny_detector(smooth)
+    region       = region_selection(edges)
+    hough        = hough_transform(region)
+    result       = draw_lane_lines(image, lane_lines(image, hough))
+    return result
+
+def process_video(test_video, output_video):
+    """
+    Read input video stream and produce a video file with detected lane lines.
+        Parameters:
+            test_video: Input video.
+            output_video: A video file with detected lane lines.
+    """
+    input_video = VideoFileClip(os.path.join('test_videos', test_video), audio=False)
+    processed = input_video.fl_image(frame_processor)
+    processed.write_videofile(os.path.join('output_videos', output_video), audio=False)
+```
+
+## **Conclusion:**
+----
+The project succeeded in detecting the lane lines clearly in the video streams.
+This project is intended to only detect (mostly) straight lines. Detecting curved lane line is behind the scope of this work.