query for ShapeFrame unlikely to work?

[vjcitn]

> getMethod("query", "ShapeFrame")
Object with tracing code, class "MethodDefinitionWithTrace"
Original definition: 
Method Definition:

function (x, ...) 
{
    args <- list(...)
    .check_bb(args)
    df <- st_as_sf(data(x))
    xy <- st_coordinates(df)
    i <- xy[, 1] >= args$xmin & xy[, 1] <= args$xmax & xy[, 2] >= 
        args$ymin & xy[, 2] <= args$ymax
    x@data <- data(x)[which(i), ]
    return(x)
}

which(i) is in the xy world, data(x) is in general not ... this might work for POINT but not for POLYGON data

[vjcitn]

I think I see a way to replace this with native sf-based calculations. More soon.

[HelenaLC]

Thanks for looking into this. Just thinking out loud: not clear to be how to best handle polygon filtering. If these are segmentation boundaries, one could imagine parts of the transcripts (points) being filtered, but a cell shouldn't be lost because one of its vertices is out of bounds... Same for pathological regions etc. Instead, one would like to insert a new edge where the bounding box (or region) intersects with existing edges? If sf can handle that elegantly, would be great!

[vjcitn]

Good questions and I think solvable. Suggestion: we step back and look at the 10 example datasets and articulate abstract use cases that can be addressed with one or more. This would be a direction of specification that I don't think I see in the python-spatialdata approaches? There are tutorials that do certain things but the "things" don't seem to be declared in advance or have specific satisfaction conditions/contracts.

[HelenaLC]

Yes, I agree that we need to verbalize concrete tasks to help guide development etc. Also start drafting some (like-to-have) workflows that put together different functionality (with TODOs). -- Could you give an example of what you had in mind? Not sure I understand your last sentence, and the issue with existing Python notebooks.

[vjcitn]

Also, is "query" the verb for this? It seems like "filter" would fit? Ignoring for the moment the possible collisions :(

[HelenaLC]

Also, is "query" the verb for this? It seems like "filter" would fit? Ignoring for the moment the possible collisions :(

I find it confusing as well. Comes from Python. They use the term "spatial query". We could also use "crop" if we like, which is semantically more similar to "filter" but avoids collision

[vjcitn]

OK, we should discuss the naming in the group. Here's what I mean about the notebooks. The MERFISH one is nice but ends fairly abruptly and seems to be improvising along the way. An important aspect of this dataset is the lack of a labels component. This one does some cropping and overlays expression over some image types/selections. The high-level goals are unclear -- are the overlays a basis for evaluating differential expression, capacity to do inference on cell type or state, etc.? The API doc is a good catalog but seems thin and super technical. In brief I think we have a good set of resources for R-based interrogation and analysis and should articulate use cases that are solved already (but may need profiling/streamlining) and others that need work at specified priorities.

[HelenaLC]

Alright. I think we're on the same page here. My feeling about the Py docs is that they are very developer-ish, i.e., technical and demonstrating the classes capacities. Instead, I'd like to see docs that focus on biology/scientific goals. As you said. Take a dataset and analyze it, demo the tasks that apply to available modalities (which may differ between platforms), and reference single-cell/spatial tasks that would be applicable at a given step. So maybe starting with workflows, and cataloging cases that we (want to) cover for different data aspects in parallel?

[vjcitn]

Yes, perhaps we can organize a call with the group to divide and conquer. BTW is the package useful for your specific work? Or are there still gaps?

[HelenaLC]

Yes, let's do that. Hm, hard to say. I think one issue is object-wide synching/metadata etc. For example, I have 8 sections and 8 single-cell samples from adjacent sections. If I want to do anything, how would the object know what comes from where/belongs together? I don't think we're addressing this. E.g., "give me only epithelia in region X from sample Y". Across all layers.

[HelenaLC]

It partly comes down to object-wide filtering based on tables (plus propagating that into spatial queries based on "what's left"), which I think Python can do already, but we just haven't got to it. And the other matter I think came up during a brainstorm session in Basel as well (that you instigated?): object-wide metadata.

[vjcitn]

That's a good use case, and I would say I don't even know how we distinguish "samples". A "donor" component might be important to add? Or some higher-level container introduced? Similarly "region" and "signature/type" need some stable vocabulary and positioning. OSCA/SingleR concepts would be a start, but even there there is a great deal of openness -- no specific contract on how to assert/assign/retrieve cell type identities/probability vectors in a mature experiment. I doubt there would be wide immediate agreement on best ways to do this but some exemplars in this domain would be very helpful.

[vjcitn]

And -- very off the cuff -- as we start thinking about writing back to the zarr store, this starts to sound more and more like transactional database concepts may need to be introduced.

[HelenaLC]

In my (perhaps naive) mind, it's just a table where rows = elements (images, labels, ...) and columns are arbitrary (e.g., technology, reference, batch, donor, treatment), that could be used to filter on based on whatever is there. Within each element, the instances (e.g., cells) are annotated by tables already. I had discussed this with Luca, and he said they are missing the former object-wide table, but it is under discussion.

[vjcitn]

Finally for now, this is a step in the direction of the cropping method I would propose but I may not get to it very soon.