Segmented polygon boundaries do not meet

[janetteeco]

SamGeo Version: 0.12.6
Python Version: 3.12
Operating System: Linux (docker container based on Ubuntu) and Windows 11

We really enjoy the package and have found that it produces great results! In working with the outputs, we have noticed that there is a buffer around the detected polygons (often one pixel wide). In our application, this results in slivers of empty or unclassified space where the segmented polygon boundaries never meet.

We have tried a number of combinations of sam_kwargs for SAM in loading the model:

    _default_parameters: dict[str, Any] = PrivateAttr(
        {
            "points_per_side": 32,
            "points_per_batch": 128,
            "pred_iou_thresh": 0.7,
            "stability_score_thresh": 0.75,
            "stability_score_offset": 0.9,
            "box_nms_thresh": 0.4,
            "crop_n_layers": 1,
            "crop_nms_thresh": 0.6,
            "crop_overlap_ratio": 512 / 1500,
            "crop_n_points_downscale_factor": 2,
            "point_grids": None,
            "min_mask_region_area": 400,
            "output_mode": "binary_mask",
        })

AND

    _default_parameters: dict[str, Any] = PrivateAttr(
        {
            "points_per_side": 32,
            "points_per_batch": 128,
            "pred_iou_thresh": 0.88,
            "stability_score_thresh": 0.95,
            "stability_score_offset": 0.0,
            "box_nms_thresh": 0.7,
            "crop_n_layers": 0,
            "crop_nms_thresh": 0.7,
            "crop_overlap_ratio": 512 / 1500,
            "crop_n_points_downscale_factor": 1,
            "point_grids": None,
            "min_mask_region_area": 0,
            "output_mode": "binary_mask",
        })

The different settings for _default_parameters do NOT close the slivers, they just change the number and size of polygons detected.

Here is the code for loading the model:

    async def load_model(self):
        device = "cuda" if torch.cuda.is_available() else "cpu"
        self._inference_model = SamGeo(
            model_type="vit_h",
            checkpoint_dir=self.inference_model_location,
            device=device,
            sam_kwargs=self._default_parameters,
        )
        return self

Note: the loading code is run on application startup. The inference_model_location contains pre-loaded sam checkpoints.

Here is the code for running the model:

    async def inner_model_run(
        self, data_input: SamGeoInputData, mask_path: str
    ) -> str:
        self._inference_model.generate(data_input.image_path, mask_path, **self._run_params)
        self._inference_model.tiff_to_vector(mask_path, data_input.shape_path)
        return data_input.shape_path

We have also tried a number of kwargs when running the code above

    _run_params: dict[str, Any] = PrivateAttr(
        {
            "batch": True,
            "foreground": True,
            "erosion_kernel": (3, 3),
            "mask_multiplier": 255,
        }
    )

AND

    _run_params: dict[str, Any] = PrivateAttr(
        {
            "batch": True,
            "foreground": False,
            "erosion_kernel": None,
            "unique": True,
        }
    )

This is an example of the type of imagery we are working with:

This is the result for the first set of run params:

Having erosion kernel set to None produced a single detected polygon within the bounding polygon.

[giswqs]

Did you check the raster masks from the output? This might be caused by the raster-to-vector conversion.

[janetteeco]

Did you check the raster masks from the output? This might be caused by the raster-to-vector conversion.

So setting the erosion kernel to None produces an all white mask. However, setting the erosion kernel to various sizes does change the size of the boundaries in the raster masks:

erosion kernel: (3,3):

erosion kernel: (5,5):

erosion kernel: (7,7):