Modifying Records

save

Save data to the database and create/update the underlying record.

Lifecycle of a Save

Before discussing the mechanics of how to save a model, it helps to understand the full lifecycle of a save operation. Of course you can ignore this lifecycle and simply use the save process to send data to a backend, but then you miss out on one of the key advantages of clearskies - supporting a state machine flow for defining your applications. The save process is controlled not just by the model but also by the columns, with equivalent hooks for both. This creates a lot of flexibility for how to control and organize an application. The overall save process looks like this:

The pre_save hook in each column is called (including the on_change_pre_save actions attached to the columns)
The pre_save hook for the model is called
The to_backend hook for each column is called and temporary data is removed from the save dictionary
The to_backend hook for the model is called
The data is persisted to the backend via a create or update call as appropriate
The post_save hook in each column is called (including the on_change_post_save actions attached to the columns)
The post_save hook in the model is called
Any data returned by the backend during the create/update operation is saved to the model along with the temporary data
The save_finished hook in each column is called (including the on_change_save_finished actions attached to the columns)
The save_finished hook in the model is called

Note that pre/post/finished hooks for all columns are called - not just the ones with data in the save. Thus, any column attached to a model can always influence the save process.

From this we can see how to use these hooks. In particular:

The pre_save hook is used to modify the data before it is persisted to the backend. This means that changes can be made to the data dictionary in the pre_save step and there will still only be a single save operation with the backend. For columns, the on_change_pre_save methods MUST be stateless - they can return data to change the save but should not make any changes themselves. This is because they may be called more than once in a given save operation.
to_backend is used to modify data on its way to the backend. Consider dates: in python these are typically represented by datetime objects but, to persist this to (for instance) an SQL database, it usually has to be converted to a string format first. That happens in the to_backend method of the datetime column.
The post_save hook is called after the backend is updated. Therefore, if you are using auto-incrementing ids, the id will only be available in ths hook. For consistency with this, clearskies doesn’t directly provide the record id until the post_save hook. If you need to make more data changes in this hook, an additional operation will be required. Since the backend has already been updated, this hook does not require a return value (and anything returned will be ignored).
The save finished hook happens after the save is fully completed. The backend is updated and the model has been updated and the model state reflects the new backend state.

The following table summarizes some key details of these hooks:

Name	Stateful	Return Value	Id Present	Backend Updated	Model Updated
`pre_save`	No	dict[str, Any]	No	No	No
`post_save`	Yes	None	Yes	Yes	No
`save_finished`	Yes	None	Yes	Yes	Yes

How to Create/Update a Model

There are two supported flows. One is to pass in a dictionary of data to save:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(user):
    user.save({
        "name": "Awesome Person",
    })
    return {"id": user.id, "name": user.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

And the other is to set new values on the columns attributes and then call save without data:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(user):
    user.name = "Awesome Person"
    user.save()
    return {"id": user.id, "name": user.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

The primray difference is that setting attributes provides strict type checking capabilities, while passing a dictionary can be done in one line. Note that you cannot combine these methods: if you set a value on a column attribute and also pass in a dictionary of data to the save, then an exception will be raised. In either case the save operation acts in place on the model object. The return value is always True - in the event of an error an exception will be raised.

If a record already exists in the model being saved, then an update operation will be executed. Otherwise, a new record will be inserted. To understand the difference yourself, you can convert a model to a boolean value - it will return True if a record has been loaded and false otherwise. You can see that with this example, where all the if statements will evaluate to True:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(user):

    if not user:
        print("We will execute a create operation")

    user.save({"name": "Test One"})
    new_id = user.id

    if user:
        print("We will execute an update operation")

    user.save({"name": "Test Two"})

    final_id = user.id

    if new_id == final_id:
        print("The id did not chnage because the second save performed an update")

    return {"id": user.id, "name": user.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

occassionaly, you may want to execute a save operation without actually providing any data. This may happen, for instance, if you want to create a record in the database that will be filled in later, and so just need an auto-generated id. By default if you call save without setting attributes on the model and without providing data to the save call, this will raise an exception, but you can make this happen with the no_data kwarg:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(user):
    # create a record with just an id
    user.save(no_data=True)

    # and now we can set the name
    user.save({"name": "Test"})

    return {"id": user.id, "name": user.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

create

Create a new record in the backend using the information in data.

The save method always operates changes the model directly rather than creating a new model instance. Often, when creating a new record, you will need to both create a new (empty) model instance and save data to it. You can do this via model.empty().save({"data": "here"}), and this method provides a simple, unambiguous shortcut to do exactly that. So, you pass your save data to the create method and you will get back a new model:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(user):
    # let's create a new record
    user.save({"name": "Alice"})

    # and now use `create` to both create a new record and get a new model instance
    bob = user.create({"name": "Bob"})

    return {
        "Alice": user.name,
        "Bob": bob.name,
    }

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

Like with save, you can set no_data=True to create a record without specifying any model data.

delete

Delete a record.

If you try to delete a record that doesn’t exist, an exception will be thrown unless you set except_if_not_exists=False. After the record is deleted from the backend, the model instance is left unchanged and can be used to fetch the data previously stored. In the following example both statements will be printed and the id and name in the “Alice” record will be returned, even though the record no longer exists:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(users):
    alice = users.create({"name": "Alice"})

    if users.find("name=Alice"):
        print("Alice exists")

    alice.delete()

    if not users.find("name=Alice"):
        print("No more Alice")

    return {"id": alice.id, "name": alice.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

model

Create a new model object and populates it with the data in data.

NOTE: the difference between this and model.create is that model.create() actually saves a record in the backend, while this method just creates a model object populated with the given data. This can be helpful if you have record data loaded up in some alternate way and want to wrap a model around it. Calling the model method does not result in any interactions with the backend.

In the following example we create a record in the backend and then make a new model instance using model, which we then use to udpate the record. The returned name will be Jane Doe.

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(users):
    jane = users.create({"name": "Jane"})

    # This effectively makes a new model instance that points to the jane record in the backend
    another_jane_object = users.model({"id": jane.id, "name": jane.name})
    # and we can perform an update operation like usual
    another_jane_object.save({"name": "Jane Doe"})

    return {"id": another_jane_object.id, "name": another_jane_object.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

empty

An alias for self.model({})

This just provides you a fresh, empty model instance that you can use for populating with data or creating a new record. Here’s a simple exmaple. Both print statements will be printed and it will return the id for the Alice record, and then null for blank_id:

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(users):
    alice = users.create({"name": "Alice"})

    if users.find("name=Alice"):
        print("Alice exists")

    blank = alice.empty()

    if not blank:
        print("Fresh instance, ready to go")

    return {"alice_id": alice.id, "blank_id": blank.id}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

is_changing

Return True/False to denote if the given column is being modified by the active save operation.

A column is considered to be changing if:

During a create operation
- It is present in the data array, even if a null value
During an update operation
- It is present in the data array and the value is changing

Note whether or not the value is changing is typically evaluated with a simple = comparison, but columns can optionally implement their own custom logic.

Pass in the name of the column to check and the data dictionary from the save in progress. This only returns meaningful results during a save, which typically happens in the pre-save/post-save hooks (either on the model class itself or in a column). Here’s an examle that extends the pre_save hook on the model to demonstrate how is_changing works:

from typing import Any, Self
import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()
    age = clearskies.columns.Integer()

    def pre_save(self: Self, data: dict[str, Any]) -> dict[str, Any]:
        if self.is_changing("name", data) and self.is_changing("age", data):
            print("My name and age have changed!")
        elif self.is_changing("name", data):
            print("Only my name is changing")
        elif self.is_changing("age", data):
            print("Only my age is changing")
        else:
            print("Nothing changed")
        return data

def my_application(users):
    jane = users.create({"name": "Jane"})
    jane.save({"age": 22})
    jane.save({"name": "Anon", "age": 23})
    jane.save({"name": "Anon", "age": 23})

    return {"id": jane.id, "name": jane.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

If you run the above example it will print out:

Only my name is changing
Only my age is changing
My name and age have changed
Nothing changed

The first message is printed out when the record is created - during a create operation, any column that is being set to a non-null value is considered to be changing. We then set the age, and since it changes from a null value (we didn’t originally set an age with the create operation, so the age was null) to a non-null value, is_changed returns True. We perform another update operation and set both name and age to new values, so both change. Finally we repeat the same save operation. This will result in another update operation on the backend, but is_changed reflects the fact that the values haven’t actually changed from their previous values.

was_changed

Return True/False to denote if a column was changed in the last save.

To emphasize, the difference between this and is_changing is that is_changing is available during the save prcess while was_changed is available after the save has finished. Otherwise, the logic for deciding if a column has changed is identical as for is_changing.

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()
    age = clearskies.columns.Integer()

def my_application(users):
    jane = users.create({"name": "Jane"})
    return {
        "name_changed": jane.was_changed("name"),
        "age_changed": jane.was_changed("age"),
    }

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

In the above example the name is changed while the age is not.

latest

Return the ‘latest’ value for a column during the save operation.

During the pre_save and post_save hooks, the model is not yet updated with the latest data. In these hooks, it’s common to want the “latest” data for the model - e.g. either the column value from the model or from the data dictionary (if the column is being updated in the save). This happens via slightly verbose lines like: data.get(column_name, getattr(self, column_name)). The latest method is just a substitue for this:

from typing import Any, Self
import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()
    age = clearskies.columns.Integer()

    def pre_save(self: Self, data: dict[str, Any]) -> dict[str, Any]:
        if not self:
            print("Create operation in progress!")
        else:
            print("Update operation in progress!")

        print("Latest name: " + str(self.latest("name", data)))
        print("Latest age: " + str(self.latest("age", data)))
        return data

def my_application(users):
    jane = users.create({"name": "Jane"})
    jane.save({"age": 25})
    return {"id": jane.id, "name": jane.name}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

The above example will print:

Create operation in progress!
Latest name: Jane
Latest age: None
Update operation in progress!
Latest name: Jane
Latest age: 25

e.g. latest returns the value in the data array (if present), the value for the column in the model, or None.

previous_value

Return the value of a column from before the most recent save.

import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()

def my_application(users):
    jane = users.create({"name": "Jane"})
    jane.save({"name": "Jane Doe"})
    return {"name": jane.name, "previous_name": jane.previous_value("name")}

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

The above example returns {"name": "Jane Doe", "previous_name": "Jane"}

If you request a key that is neither a column nor was present in the previous data array, then you’ll receive a key error. You can suppress this by setting silent=True in your call to previous_value.

post_save

A hook to add additional logic in the post-save step of the save process.

It is passed in the data being saved as well as the id of the record. Keep in mind that the post save hook happens after the backend has been updated (but before the model is updated) so if you need to make any changes to the backend you must execute another save operation. Since the backend is already updated, the return value from this function is ignored (it should return None):

from typing import Any, Self
import clearskies

class History(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    message = clearskies.columns.String()
    created_at = clearskies.columns.Created(date_format="%Y-%m-%d %H:%M:%S.%f")

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()
    histories = clearskies.di.inject.ByClass(History)

    id = clearskies.columns.Uuid()
    age = clearskies.columns.Integer()
    name = clearskies.columns.String()

    def post_save(self: Self, data: dict[str, Any], id: str | int) -> None:
        if not self.is_changing("age", data):
            return

        name = self.latest("name", data)
        age = self.latest("age", data)
        self.histories.create({"message": f"My name is {name} and I am {age} years old"})

def my_application(users, histories):
    jane = users.create({"name": "Jane"})
    jane.save({"age": 25})
    jane.save({"age": 26})
    jane.save({"age": 30})

    return [history.message for history in histories.sort_by("created_at", "ASC")]

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User, History],
)
cli()

pre_save

A hook to add additional logic in the pre-save step of the save process.

The pre/post/finished steps of the model are directly analogous to the pre/post/finished steps for the columns.

pre-save is inteneded to be a stateless hook (e.g. you should not make changes to the backend) where you can adjust the data being saved to the model. It is called before any data is persisted to the backend and must return a dictionary of data that will be added to the save, potentially over-writing the save data. Since pre-save happens before communicating with the backend, the record itself will not yet exist in the event of a create operation, and so the id will not be-present for auto-incrementing ids. As a result, the record id is not provided during the pre-save hook. See the breakdown of the save lifecycle in the save documentation above for more details.

An here’s an example of using it to set some additional data during a save:

from typing import Any, Self
import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()
    is_anonymous = clearskies.columns.Boolean()

    def pre_save(self: Self, data: dict[str, Any]) -> dict[str, Any]:
        additional_data = {}

        if self.is_changing("name", data):
            additional_data["is_anonymous"] = not bool(data["name"])

        return additional_data

def my_application(users):
    jane = users.create({"name": "Jane"})
    is_anonymous_after_create = jane.is_anonymous

    jane.save({"name":""})
    is_anonymous_after_first_update = jane.is_anonymous

    jane.save({"name": "Jane Doe"})
    is_anonymous_after_last_update = jane.is_anonymous

    return {
        "is_anonymous_after_create": is_anonymous_after_create,
        "is_anonymous_after_first_update": is_anonymous_after_first_update,
        "is_anonymous_after_last_update": is_anonymous_after_last_update,
    }

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User],
)
cli()

In our pre-save hook we set the is_anonymous field to either True or False depending on whether or not there is a value in the incoming name column. As a result, after the original create operation (when the name is "Jane", is_anonymous is False. We then update the name and set it to an empty string, and is_anonymous becomes True. We then update one last time to set a name again and is_anonymous becomes False.

save_finished

A hook to add additional logicin the save_finished step of the save process.

It has no retrun value and is passed no data. By the time this fires the model has already been updated with the new data. You can decide on the necessary actions using the was_changed and the previous_value functions.

from typing import Any, Self
import clearskies

class History(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()

    id = clearskies.columns.Uuid()
    message = clearskies.columns.String()
    created_at = clearskies.columns.Created(date_format="%Y-%m-%d %H:%M:%S.%f")

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()
    histories = clearskies.di.inject.ByClass(History)

    id = clearskies.columns.Uuid()
    age = clearskies.columns.Integer()
    name = clearskies.columns.String()

    def save_finished(self: Self) -> None:
        if not self.was_changed("age"):
            return

        self.histories.create({"message": f"My name is {self.name} and I am {self.age} years old"})

def my_application(users, histories):
    jane = users.create({"name": "Jane"})
    jane.save({"age": 25})
    jane.save({"age": 26})
    jane.save({"age": 30})

    return [history.message for history in histories.sort_by("created_at", "ASC")]

cli = clearskies.contexts.Cli(
    my_application,
    classes=[User, History],
)
cli()

where_for_request

A hook to automatically apply filtering whenever the model makes an appearance in a get/update/list/search handler.

Note that this automatically affects the behavior of the various list endpoints, but won’t be called when you create your own queries directly. Here’s an example where the model restricts the list endpoint so that it only returns users with an age over 18:

from typing import Any, Self
import clearskies

class User(clearskies.Model):
    id_column_name = "id"
    backend = clearskies.backends.MemoryBackend()
    id = clearskies.columns.Uuid()
    name = clearskies.columns.String()
    age = clearskies.columns.Integer()

    def where_for_request(
        self: Self,
        model: Self,
        input_output: Any,
        routing_data: dict[str, str],
        authorization_data: dict[str, Any],
        overrides: dict[str, clearskies.Column] = {},
    ) -> Self:
        return model.where("age>=18")

list_users = clearskies.endpoints.List(
    model_class=User,
    readable_column_names=["id", "name", "age"],
    sortable_column_names=["id", "name", "age"],
    default_sort_column_name="name",
)

wsgi = clearskies.contexts.WsgiRef(
    list_users,
    classes=[User],
    bindings={
        "memory_backend_default_data": [
            {
                "model_class": User,
                "records": [
                    {"id": "1-2-3-4", "name": "Bob", "age": 20},
                    {"id": "1-2-3-5", "name": "Jane", "age": 17},
                    {"id": "1-2-3-6", "name": "Greg", "age": 22},
                ],
            },
        ]
    },
)
wsgi()