Find and Filter

Insert Some Data

We will work on the existing Player model.

Let's create some data in the database test_db and a collection player.

# Code omitted above

class Player(Document):
    name: str
    country_code: str
    rating: Optional[int] = None

    class ODMConfig(Document.ODMConfig):
        indexes = [
            IndexModel([("rating", ASCENDING)]),
        ]


def create_documents():
    Player(name="Pelé", country_code="BRA", rating=98).create()
    Player(name="Diego Maradona", country_code="ARG", rating=97).create()
    Player(name="Zinedine Zidane", country_code="FRA", rating=94).create()
    Player(name="Ronaldo", country_code="BRA", rating=94).create()
    Player(name="Neymar", country_code="BRA", rating=89).create()
    Player(name="Lionel Messi", country_code="ARG", rating=91).create()
    Player(name="Ángel Di María", country_code="ARG", rating=84).create()
    Player(name="Karim Benzema", country_code="FRA", rating=89).create()
    Player(name="Antoine Griezmann", country_code="FRA", rating=85).create()
    Player(name="Kylian Mbappé", country_code="FRA", rating=91).create()
    Player(name="Gerd Müller", country_code="GER").create()
    Player(name="Miroslav Klose", country_code="GER", rating=91).create()
    Player(name="Thomas Müller", country_code="GER", rating=87).create()
    Player(name="Cristiano Ronaldo", country_code="POR", rating=87).create()
    Player(name="Eusébio", country_code="POR", rating=93).create()
    Player(name="Diogo Jota", country_code="POR", rating=85).create()
    Player(name="David Beckham", country_code="ENG", rating=89).create()
    Player(name="Wayne Rooney", country_code="ENG", rating=80).create()
    Player(name="Harry Kane", country_code="ENG", rating=89).create()

# Code omitted below

Full file preview

import os
from typing import Optional

from mongodb_odm import ASCENDING, Document, IndexModel, apply_indexes, connect


class Player(Document):
    name: str
    country_code: str
    rating: Optional[int] = None

    class ODMConfig(Document.ODMConfig):
        indexes = [
            IndexModel([("rating", ASCENDING)]),
        ]


def create_documents():
    Player(name="Pelé", country_code="BRA", rating=98).create()
    Player(name="Diego Maradona", country_code="ARG", rating=97).create()
    Player(name="Zinedine Zidane", country_code="FRA", rating=94).create()
    Player(name="Ronaldo", country_code="BRA", rating=94).create()
    Player(name="Neymar", country_code="BRA", rating=89).create()
    Player(name="Lionel Messi", country_code="ARG", rating=91).create()
    Player(name="Ángel Di María", country_code="ARG", rating=84).create()
    Player(name="Karim Benzema", country_code="FRA", rating=89).create()
    Player(name="Antoine Griezmann", country_code="FRA", rating=85).create()
    Player(name="Kylian Mbappé", country_code="FRA", rating=91).create()
    Player(name="Gerd Müller", country_code="GER").create()
    Player(name="Miroslav Klose", country_code="GER", rating=91).create()
    Player(name="Thomas Müller", country_code="GER", rating=87).create()
    Player(name="Cristiano Ronaldo", country_code="POR", rating=87).create()
    Player(name="Eusébio", country_code="POR", rating=93).create()
    Player(name="Diogo Jota", country_code="POR", rating=85).create()
    Player(name="David Beckham", country_code="ENG", rating=89).create()
    Player(name="Wayne Rooney", country_code="ENG", rating=80).create()
    Player(name="Harry Kane", country_code="ENG", rating=89).create()


def read_documents():
    players = Player.find()
    for player in players:
        print(player)
    print()


def filter_documents():
    documents = Player.find(filter={Player.rating: {"$gte": 90}})
    for player in documents:
        print(player)
    print()


def read_documents_with_projection():
    documents = Player.find(projection={Player.rating: 0})
    for player in documents:
        print(player)
    print()


def main():
    connect(os.environ.get("MONGO_URL", "mongodb://localhost:27017/testdb"))
    apply_indexes()
    create_documents()

    read_documents()
    filter_documents()
    read_documents_with_projection()


if __name__ == "__main__":
    main()

The data will look like

{
  "_id": ObjectId('id'),
  "name": "Pelé",
  "country_code": "BRA",
  "rating": null
}
{
  "_id": ObjectId('id'),
  "name": "Diego Maradona",
  "country_code": "ARG",
  "rating": 97
}
{
  "_id": ObjectId('id'),
  "name": "Zinedine Zidane",
  "country_code": "FRA",
  "rating": 96
}
...

The rating field should be set as an index in the database as we set it as an index in ODMConfig.

We will continue from the last code that we used to create some data.

Find data

Find Data Using MongoDB Console

Let's read the player collection using the MongoDB console.

use test_db

db.player.find()

Run the above code in the MongoDB console. We should get all the data that was previously created.

Find Data With MongoDB-ODM

We will use the find method of the Player class that is implemented in the Document class.

# Code omitted above

def read_documents():
    players = Player.find()
    for player in players:
        print(player)
    print()

# Code omitted below

Full file preview

import os
from typing import Optional

from mongodb_odm import ASCENDING, Document, IndexModel, apply_indexes, connect


class Player(Document):
    name: str
    country_code: str
    rating: Optional[int] = None

    class ODMConfig(Document.ODMConfig):
        indexes = [
            IndexModel([("rating", ASCENDING)]),
        ]


def create_documents():
    Player(name="Pelé", country_code="BRA", rating=98).create()
    Player(name="Diego Maradona", country_code="ARG", rating=97).create()
    Player(name="Zinedine Zidane", country_code="FRA", rating=94).create()
    Player(name="Ronaldo", country_code="BRA", rating=94).create()
    Player(name="Neymar", country_code="BRA", rating=89).create()
    Player(name="Lionel Messi", country_code="ARG", rating=91).create()
    Player(name="Ángel Di María", country_code="ARG", rating=84).create()
    Player(name="Karim Benzema", country_code="FRA", rating=89).create()
    Player(name="Antoine Griezmann", country_code="FRA", rating=85).create()
    Player(name="Kylian Mbappé", country_code="FRA", rating=91).create()
    Player(name="Gerd Müller", country_code="GER").create()
    Player(name="Miroslav Klose", country_code="GER", rating=91).create()
    Player(name="Thomas Müller", country_code="GER", rating=87).create()
    Player(name="Cristiano Ronaldo", country_code="POR", rating=87).create()
    Player(name="Eusébio", country_code="POR", rating=93).create()
    Player(name="Diogo Jota", country_code="POR", rating=85).create()
    Player(name="David Beckham", country_code="ENG", rating=89).create()
    Player(name="Wayne Rooney", country_code="ENG", rating=80).create()
    Player(name="Harry Kane", country_code="ENG", rating=89).create()


def read_documents():
    players = Player.find()
    for player in players:
        print(player)
    print()


def filter_documents():
    documents = Player.find(filter={Player.rating: {"$gte": 90}})
    for player in documents:
        print(player)
    print()


def read_documents_with_projection():
    documents = Player.find(projection={Player.rating: 0})
    for player in documents:
        print(player)
    print()


def main():
    connect(os.environ.get("MONGO_URL", "mongodb://localhost:27017/testdb"))
    apply_indexes()
    create_documents()

    read_documents()
    filter_documents()
    read_documents_with_projection()


if __name__ == "__main__":
    main()

In the read_players function, we call the find method.

The find is a classmethod that will return a generator. We will be able to iterate over the generator by using a loop.

We can also use the next function like next(players). But next(players) may raise an error if there is no data in the player collection.

Then we iterate through players.

Each object in players is equivalent to a Player object. And we should get all the functionality of the Player class.

Check the console after executing the function.

Player(id=ObjectId('id'), name='Pelé', country_code='BRA', rating=98, _id=ObjectId('id'))
Player(id=ObjectId('id'), name='Diego Maradona', country_code='ARG', rating=97, _id=ObjectId('id'))
Player(id=ObjectId('id'), name='Zinedine Zidane', country_code='FRA', rating=94, _id=ObjectId('id'))
...

For each object, we should get type hints that are provided by standard Python and Pydantic.

Accessing data from the returned object

To access values from the object we will use the . operator. Nothing special, just standard Python.

Example:

id = player._id
name = player.name
country_code = player.country_code
rating = player.rating

Filter data

We will filter all players where the rating is greater than or equal to 10.

Filtering Data Using MongoDB Console

Let's filter the player collection using the MongoDB console.

use test_db

db.player.find({"rating": {"$gte": 10}})

After executing the above bash code in the MongoDB console, we should get all players who have a rating greater than or equal to 10.

Filter Data with MongoDB-ODM

We will use the same find method of the Player class that we used previously to read all data.

The find method accepts several arguments like a filter, sort, limit, etc.

# Code omitted above

def filter_documents():
    documents = Player.find(filter={Player.rating: {"$gte": 90}})
    for player in documents:
        print(player)
    print()

# Code omitted below

Full file preview

import os
from typing import Optional

from mongodb_odm import ASCENDING, Document, IndexModel, apply_indexes, connect


class Player(Document):
    name: str
    country_code: str
    rating: Optional[int] = None

    class ODMConfig(Document.ODMConfig):
        indexes = [
            IndexModel([("rating", ASCENDING)]),
        ]


def create_documents():
    Player(name="Pelé", country_code="BRA", rating=98).create()
    Player(name="Diego Maradona", country_code="ARG", rating=97).create()
    Player(name="Zinedine Zidane", country_code="FRA", rating=94).create()
    Player(name="Ronaldo", country_code="BRA", rating=94).create()
    Player(name="Neymar", country_code="BRA", rating=89).create()
    Player(name="Lionel Messi", country_code="ARG", rating=91).create()
    Player(name="Ángel Di María", country_code="ARG", rating=84).create()
    Player(name="Karim Benzema", country_code="FRA", rating=89).create()
    Player(name="Antoine Griezmann", country_code="FRA", rating=85).create()
    Player(name="Kylian Mbappé", country_code="FRA", rating=91).create()
    Player(name="Gerd Müller", country_code="GER").create()
    Player(name="Miroslav Klose", country_code="GER", rating=91).create()
    Player(name="Thomas Müller", country_code="GER", rating=87).create()
    Player(name="Cristiano Ronaldo", country_code="POR", rating=87).create()
    Player(name="Eusébio", country_code="POR", rating=93).create()
    Player(name="Diogo Jota", country_code="POR", rating=85).create()
    Player(name="David Beckham", country_code="ENG", rating=89).create()
    Player(name="Wayne Rooney", country_code="ENG", rating=80).create()
    Player(name="Harry Kane", country_code="ENG", rating=89).create()


def read_documents():
    players = Player.find()
    for player in players:
        print(player)
    print()


def filter_documents():
    documents = Player.find(filter={Player.rating: {"$gte": 90}})
    for player in documents:
        print(player)
    print()


def read_documents_with_projection():
    documents = Player.find(projection={Player.rating: 0})
    for player in documents:
        print(player)
    print()


def main():
    connect(os.environ.get("MONGO_URL", "mongodb://localhost:27017/testdb"))
    apply_indexes()
    create_documents()

    read_documents()
    filter_documents()
    read_documents_with_projection()


if __name__ == "__main__":
    main()

Warning

There is no validation of JSON objects. So make sure every keyword is correct. And double-check spelling mistakes.

The find class method will return an iterator.

The returned data should be the same as the returned data in MongoDB Console.

Check the console after executing the code.

Player(id=ObjectId('id'), name='Lionel Messi', country_code='ARG', rating=91, _id=ObjectId('id'))
Player(id=ObjectId('id'), name='Kylian Mbappé', country_code='FRA', rating=91, _id=ObjectId('id'))
Player(id=ObjectId('id'), name='Miroslav Klose', country_code='GER', rating=91, _id=ObjectId('id'))
...

Read data with projection

Sometimes we need to limit some fields that should not be pulled from the database to improve network performance. For example, we have a description field that has a very long string but on the list page, we don't need to pull it.

In that scenario, we can use projection kwargs in the find function to limit the data pulling.

In this example, we eliminate the rating field while getting data from the database.

# Code omitted above

def read_documents_with_projection():
    documents = Player.find(projection={Player.rating: 0})
    for player in documents:
        print(player)
    print()

# Code omitted below

Full file preview

import os
from typing import Optional

from mongodb_odm import ASCENDING, Document, IndexModel, apply_indexes, connect


class Player(Document):
    name: str
    country_code: str
    rating: Optional[int] = None

    class ODMConfig(Document.ODMConfig):
        indexes = [
            IndexModel([("rating", ASCENDING)]),
        ]


def create_documents():
    Player(name="Pelé", country_code="BRA", rating=98).create()
    Player(name="Diego Maradona", country_code="ARG", rating=97).create()
    Player(name="Zinedine Zidane", country_code="FRA", rating=94).create()
    Player(name="Ronaldo", country_code="BRA", rating=94).create()
    Player(name="Neymar", country_code="BRA", rating=89).create()
    Player(name="Lionel Messi", country_code="ARG", rating=91).create()
    Player(name="Ángel Di María", country_code="ARG", rating=84).create()
    Player(name="Karim Benzema", country_code="FRA", rating=89).create()
    Player(name="Antoine Griezmann", country_code="FRA", rating=85).create()
    Player(name="Kylian Mbappé", country_code="FRA", rating=91).create()
    Player(name="Gerd Müller", country_code="GER").create()
    Player(name="Miroslav Klose", country_code="GER", rating=91).create()
    Player(name="Thomas Müller", country_code="GER", rating=87).create()
    Player(name="Cristiano Ronaldo", country_code="POR", rating=87).create()
    Player(name="Eusébio", country_code="POR", rating=93).create()
    Player(name="Diogo Jota", country_code="POR", rating=85).create()
    Player(name="David Beckham", country_code="ENG", rating=89).create()
    Player(name="Wayne Rooney", country_code="ENG", rating=80).create()
    Player(name="Harry Kane", country_code="ENG", rating=89).create()


def read_documents():
    players = Player.find()
    for player in players:
        print(player)
    print()


def filter_documents():
    documents = Player.find(filter={Player.rating: {"$gte": 90}})
    for player in documents:
        print(player)
    print()


def read_documents_with_projection():
    documents = Player.find(projection={Player.rating: 0})
    for player in documents:
        print(player)
    print()


def main():
    connect(os.environ.get("MONGO_URL", "mongodb://localhost:27017/testdb"))
    apply_indexes()
    create_documents()

    read_documents()
    filter_documents()
    read_documents_with_projection()


if __name__ == "__main__":
    main()

Check the console after executing the code.

Player(id=ObjectId('id'), name='Zinedine Zidane', country_code='FRA', rating=None, _id=ObjectId('id'))
Player(id=ObjectId('id'), name='Ronaldo', country_code='BRA', rating=None, _id=ObjectId('id'))
Player(id=ObjectId('id'), name='Lionel Messi', country_code='ARG', rating=None, _id=ObjectId('id'))
...

Warning

We need to pull all fields that are required. We can only eliminate optional or nullable fields. If we do not pull a field, then the field value will be the default value.

Model Field and ID Access

For better type safety, error prevention, and IDE support, mongodb-odm supports accessing field names directly through the model class. Field name as a string literal is prone to typos and lacks type safety. You can use the model class attributes directly to access field names.

The value of Player.name is equivalent to name (Player.id is as exception). We can use Player.name instead of "name". This will provide better type safety and IDE support.

We can use the model field (Player.name) anywhere we want to.

For example:

• We can use it in the find method

Player.find(filter={Player.rating: {"$gte": 90}})

• We can also use the model field (Player.name) in the sort method.

Player.find(sort=[(Player.rating, ASCENDING)])

• We can also use the model field (Player.name) in the update_one method.

Player.update_one(filter={Player.rating: {"$gte": 90}}, update={Player.rating: 90})

Warning

We have exception here for the id field. The value of Player.id is equivalent to _id. Since MongoDB uses _id as the default field name for the primary key, we use id as the field name in the model class.

Learning Curve

Filtering data is a crucial part of most DBMS systems.

To filter data from a database, every developer should know how a particular database provides the filtering API.

Also, when we add an extra modeling system for a better developer experience, things get more complicated. Then developers should know both the filtering system (database way and model way). But we won't do that here.

MongoDB already has a broad JSON query system. We will use that directly.

So no complex query learning curve was added to the learning process. We will filter data the MongoDB way.