Scheduled Messages
The messaging framework supports scheduling messages to be sent on a one-time or recurring basis.
It uses a queuing architecture similar to the SMS framework, to make it easier to scale reminders processing power horizontally.
An earlier incarnation of this framework was called “reminders”, so some code references to reminders remain, such
as the reminder_queue
.
Definitions
Scheduled messages are represented in the UI as “broadcasts” and “conditional alerts.”
Broadcasts, represented by the subclasses of corehq.messaging.scheduling.models.abstract.Broadcast, allow configuring a recurring schedule to send a particular message type and content to a particular set of recipients.
Conditional alerts, represented by corehq.apps.data_interfaces.models.AutomaticUpdateRule, contain a similar recurring schedule but act on cases. They are configured to trigger on when cases meet a set of criteria, such as a case property changing to a specific value.
The two models share much of their code. This document primarily addresses conditional alerts and will refer to them as “rules,” as most of the code does.
A rule definition, defines the rules for:
what criteria cause a reminder to be triggered
when the message should send once the criteria are fulfilled
who the message should go to
on what schedule and frequency the message should continue to be sent
the content to send
what causes the rule to stop
Conditional Alerts / Case Update Rules
A conditional alert, represented by corehq.apps.data_interfaces.models.AutomaticUpdateRule, defines an instance of a rule definition and keeps track of the state of the rule instance throughout its lifetime.
For example, a conditional alert definition may define a rule for sending an SMS to a case of type patient
, and
sending an SMS appointment reminder to the case 2 days before the case’s appointment_date
case property.
As soon as a case is created or updated in the given project to meet the criteria of having type patient
and having an appointment_date
, the framework will create a reminder instance to track it.
After the message is sent 2 days before the appointment_date
, the rule instance is deactivated
to denote that it has completed the defined schedule and should not be sent again.
In order to keep messaging responsive to case changes, every time a case is saved, the corehq.messaging.tasks.sync_case_for_messaging function is called to handle any changes. This is controlled via case-pillow.
Similarly, any time a rule is updated, a corehq.messaging.tasks.run_messaging_rule task is spawned to rerun it against all cases in the project.
The aim of the framework is to always be completely responsive to all changes. So in the example above,
if a case’s appointment_date
changes before the appointment reminder is actually sent, the framework will
update the schedule instance (more on these below) automatically in order to reflect the new appointment date. And if the
appointment reminder went out months ago but a new appointment_date
value is given to the case for a new
appointment, the same instance is updated again to reflect a new message that must go out.
Similarly, if the rule definition is updated to use a different case property other than appointment_date
,
all existing schedule instances are deleted and any new ones are created if they meet the criteria.
Lifecycle of a Rule
As mentioned above, whe a rule is changed, all cases of the relevant type in the domain are re-processed. The steps of this process are as follows:
When a conditional alert is created or activated, a corehq.messaging.tasks.initiate_messaging_rule_run task is spawned.
This locks the rule, so that it cannot be edited from the UI, and spawns a corehq.messaging.tasks.run_messaging_rule task.
This task spawns a corehq.messaging.tasks.sync_case_for_messaging_rule task for every case of the rule’s case type. It also adds a corehq.messaging.tasks.set_rule_complete task to unlock the rule when all of the
sync_case
tasks are finished.This task calls corehq.apps.data_interfaces.models.AutomaticUpdateRule.run_rule on its case.
run_rule
checks whether or not the case meets the rule’s criteria and acts accordingly. When the case matches, this callsrun_actions_when_case_matches
and thenwhen_case_matches
. Conditional alert actions useCreateScheduleInstanceActionDefinition
which implementswhen_case_matches
to call corehq.messaging.scheduling.tasks.refresh_case_alert_schedule_instances or corehq.messaging.scheduling.tasks.refresh_case_timed_schedule_instances depending on whether the rule is immediate or scheduled.The refresh functions act on subclasses of corehq.messaging.scheduling.tasks.ScheduleInstanceRefresher, which create, update, and delete “schedule instance” objects, which are subclasses of corehq.messaging.scheduling.scheduling_partitioned.models.ScheduleInstance. These schedule instances track their schedule, recipients, and state relating to their next event. They are processed by a queue (see next section).
Queueing
All of the schedule instances in the database represent the queue of messages that should be sent. The way a schedule instance is processed is as follows:
The polling process (
python manage.py queue_schedule_instances
), which runs as a supervisor process on one of the celery machines, constantly polls for schedules that should be processed by querying for schedule instances that have anext_event_due
property that is in the past.Once a schedule instance that needs to be processed has been identified, the framework spawns one of several tasks from corehq.messaging.scheduling.tasks to handle it. These tasks include
handle_alert_schedule_instance
,handle_timed_schedule_instance
,handle_case_alert_schedule_instance
, andhandle_case_timed_schedule_instance
.The handler looks at the schedule instances and instructs it to 1) take the appropriate action that has been configured (for example, send an sms), and 2) update the state of the instance so that it gets scheduled for the next action it must take based on the reminder definition. This is handled by corehq.messaging.scheduling.scheduling_partitioned.models.ScheduleInstance.handle_current_event
A second queue (python manage.py run_sms_queue
), which is set up similarly on each celery machine that consumes
from the reminder_queue
,handles the sending of messages.
Event Handlers
A rule (or broadcast) sends content of one type. At the time of writing, the content a reminder definition can be configured to send includes:
SMS
SMS Survey
Emails
Push Notifications
In the case of SMS SurveysSessions, the survey content is defined using a form in an app which is then played to the recipients over SMS or Whatsapp.