feat(NODE-7320): Update $ne operator to include null type

[christophehurpeau]

Description

Summary of Changes

Allow $ne operator to accept null values.

null value is useful when we want to have a similar behavior as exists, but including null values even if the Value is not declared as nullable.

Notes for Reviewers

What is the motivation for this change?

Release Highlight

Release notes highlight

Double check the following

• Lint is passing (npm run check:lint)
• Self-review completed using the steps outlined here
• PR title follows the correct format: type(NODE-xxxx)[!]: description
  • Example: feat(NODE-1234)!: rewriting everything in coffeescript
• Changes are covered by tests
• New TODOs have a related JIRA ticket

[tadjik1]

Hi @christophehurpeau, thank you for the contribution! We are going to discuss this change internally as there are some usability and necessity questions arose. Can you maybe share the exact use-case or small snippet to demonstrate usage scenario, this would be extremely helpful.

[christophehurpeau]

@tadjik1

Of course.

Most of our models include an schema with non-nullable fields. Here a simple example:

interface EndUser {
  _id: Types.ObjectID;
  fullname: string;
  email?: string;
}

With an index on email.
When we want to query all enduser with existing emails:

const endUsers = await EndUserQueries.find({ email: { $ne: null } })

[tadjik1]

Hi @christophehurpeau,
thanks for providing that snippet - it makes the context and the goal of this change much clearer.

While I understand the convenience this would provide from the application side, we have to consider the driver’s role as a transparent interface to the database. There is a fundamental distinction between a field being null and a field being missing.
Since your schema defines email as optional (essentially string | undefined), the idiomatic way to query for this in MongoDB is using $exists, for example { email: { $exists: true } } - to find all users with email.
If you do have users with their emails null in the collection, it might be best to reflect that in your interface:

interface EndUser {
  ...
  email: string | null;
}

Please let me know if this helps!

[christophehurpeau]

Hi @christophehurpeau, thanks for providing that snippet - it makes the context and the goal of this change much clearer.

While I understand the convenience this would provide from the application side, we have to consider the driver’s role as a transparent interface to the database. There is a fundamental distinction between a field being null and a field being missing. Since your schema defines email as optional (essentially string | undefined), the idiomatic way to query for this in MongoDB is using $exists, for example { email: { $exists: true } } - to find all users with email. If you do have users with their emails null in the collection, it might be best to reflect that in your interface:

interface EndUser {
  ...
  email: string | null;
}

Please let me know if this helps!

From what I understood, $exists did not use indexes. that's why we did not use it. It seems the bug was fixed, or for some other reason it works now, at least in the case I tested. I guess we can use $exists then. Thank you for your response.