Separate table change observation from querying

[m6s]

Add a method Observable<List<String>> observeTables(String... tables) or similar to BriteDatabase, that notifies observers with a set of affected tables, but does not create any query.

Scenario: We sync cats and dogs stored on our backend, and persist them using a transaction.(pseudo-code, cursor/cv mapping omitted)

try (Transaction transaction = database.newTransaction()) {
    for(Cat cat: cats) {
        database.insert("cat", cat);
    }
    for(Dog dog: dogs) {
        database.insert("dog", dog);
    }
    transaction.markSuccessful();
}

We fetch all cats and dogs when initializing our view. (In real code in a background thread, obviously)

List cats = database.query("SELECT * FROM cat");
List dogs = database.query("SELECT * FROM dog");
catAndDogView.setCatsAndDogs(cats, dogs);

How do I make this reactive?

Observable<List> cats = database.createQuery("cat", "SELECT * FROM cat");
Observable<List> dogs = database.createQuery("dog", "SELECT * FROM dog");
Observable.combineLatest(cats, dogs, (cats, dogs) -> new Object[]{cats, dogs})
    .subscribe(objects -> catAndDogView.setCatsAndDogs(objects[0], objects[1]));

This is flawed, because I'm now getting two updates per transaction. Replacing combineLatest() with zip() won't do neither, because then I'm missing out on updates that involve only cats, or only dogs. A work-around is having each query triggered by changes on either table, but that comes at the expense of running some redundant queries.

As an added benefit of my proposal, having the opportunity to react to table changes myself, would allow me to create DAOs with synchronous methods, and reusing these methods when I want to create reactive versions.

class DAO {
    List<Cat> findBlackCats(BriteDatabase database) {
        return database.query("SELECT * FROM cat WHERE color = black");
    }
}

Observable<List<Cat>> blackCats = database.observeTables("cat")
    .map(ignore -> dao.findBlackCats(database));

My proposal is based on what SquiDB seems to offer.

[JakeWharton]

Seems reasonable. You make a good argument. Just with using Set (since we
have that observable already internally).

Let me think on it for another day or three.

On Sat, Dec 26, 2015, 4:46 PM Matt notifications@github.com wrote:

Add a method Observable<List> observeTables(String... tables) or
similar to BriteDatabase, that notifies observers with a set of affected
tables, but does not create any query.

Scenario: We sync cats and dogs stored on our backend, and persist them
using a transaction.(pseudo-code, cursor/cv mapping omitted)

try (Transaction transaction = database.newTransaction()) {
for(Cat cat: cats) {
database.insert("cat", cat);
}
for(Dog dog: dogs) {
database.insert("dog", dog);
}
transaction.markSuccessful();
}

We fetch all cats and dogs when initializing our view. (In real code in a
background thread, obviously)

List cats = database.query("SELECT * FROM cat");List dogs = database.query("SELECT * FROM dog");
catAndDogView.setCatsAndDogs(cats, dogs);

How do I make this reactive?

Observable cats = database.createQuery("cat", "SELECT * FROM cat");Observable dogs = database.createQuery("dog", "SELECT * FROM dog");Observable.combineLatest(cats, dogs, (cats, dogs) -> new Object[]{cats, dogs})
.subscribe(objects -> catAndDogView.setCatsAndDogs(objects[0], objects[1]));

This is flawed, because I'm now getting two updates per transaction.
Replacing combineLatest() with zip() won't do neither, because then I'm
missing out on updates that involve only cats, or only dogs. A work-around
is having each query triggered by changes on either table, but that comes
at the expense of running some redundant queries.

As an added benefit of my proposal, having the opportunity to react to
table changes myself, would allow me to create DAOs with synchronous
methods, and reusing these methods when I want to create reactive versions.

class DAO {
List findBlackCats(BriteDatabase database) {
return database.query("SELECT * FROM cat WHERE color = black");
}
}
Observable<List> blackCats = database.observeTables("cat")
.map(ignore -> database.findBlackCats(database));

My proposal is based on what SquiDB
https://github.com/yahoo/squidb/wiki/Observing-with-RxJava seems to
offer.

—
Reply to this email directly or view it on GitHub
#80.

[JakeWharton]

Is exposing Observable<Set<String>> tableTriggers() enough for you?

[JakeWharton]

because I'm now getting two updates per transaction

By the way, a simple but alternate way to avoid this problem would be to just use one of the throttle operators like throttleLast(100, MILLISECONDS) to avoid spammy updates between your combineLatest and subscribe.

[m6s]

Exposing Observable<Set<String>> tableTriggers() is enough for me: By doing so, you provide all the information that is available, to consumers of your API, instead of doing some pre-filtering. You have to decide, though, if the availability of such a Set of changed tables is an implementation detail or not.

Throttling is a work-around, but it feels hackish. The duration to choose is a bit arbitrary and would depend on the number of triggered queries (application-wide) and on how long each of those queries takes in the worst case.

I'm not asking because I'm having a specific need right now. I do have, but it's similar to the scenario that I described, and I wouldn't be bothered by a view updating twice (or more often). I thought it would be more problematic if I was sending out emails upon table changes.

[maxandron]

Jumping in here, I have a use case where the feature Matt proposed would be very useful.
I have several situations in my code where I need to create a query and use that data immediately, having an observable in that case instead of parsing the cursor myself would be very helpful.

The problem here is that I can't use something like toBlocking because I'm subscribed to a PublishSubject that will never finish.

A way to solve this would be to create a method that creates a new observable from the Query and does not use it as part of the triggers publisher.

I could of course also use .timeout(arbitraryNumber, MILLISECONDS), but as Matt mentions it feels hackish and error prone.

[gabrielittner]

Would you accept a pull request to expose the internal PublishSubject as Observable<Set<String>>?

[JakeWharton]

It wasn't that easy when I looked at this a few months back, but I can't remember why.

…

On Sat, Dec 31, 2016 at 6:22 AM Gabriel Ittner ***@***.***> wrote: Would you accept a pull request to expose the internal PublishSubject as Observable<Set<String>>? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#80 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEEETHCGxUFhBSu6l7jPQQlvjXbXi3Dks5rNjrvgaJpZM4G7gCp> .

[JakeWharton]

For now you can SELECT 0 as your query, skip(1), and ignore the emitted value. Without more investigation I'm going to table adding support for this natively.