How Data Mapping Works
DHIS2-, OpenMRS- and FHIR Integration all use the ValueSource class to map CommCare data to API resources.
A ValueSource is given in JSON format. e.g.
{
"case_property": "active",
"jsonpath": "$.active"
}
This ValueSource maps the value from the case property named “active” to the “active” property of an API resource.
Different Sources of Values
The ValueSource class supports several different sources of values:
case_property
: As seen above, a ValueSource can be used for fetching a value from a case property, or setting a value on a case property.form_question
: Fetches a value from a form question. e.g. “/data/foo/bar” will get the value of a form question named “bar” in the group “foo”. Form metadata is also available, e.g. “/metadata/received_on” is the server time when the form submission was received. You can find more details in the source code at corehq.motech.value_source:FormQuestioncase_owner_ancestor_location_field
: Specifies a location metadata field name. The ValueSource will start at the location of the case owner, traverse up their location hierarchy, and return the first value it finds for a location with that field. This can be used for mapping CommCare locations to locations or organization units in a remote system.form_user_ancestor_location_field
: Specifies a location metadata field name. Similar to case_owner_ancestor_location_field but for forms instead of cases. The ValueSource will start at the location of the user who submitted the form, traverse up their location hierarchy, and return the first value it finds for a location with that field. This can be used for mapping CommCare locations to locations or organization units in a remote system.subcase_value_source
: Defines a ValueSource to be evaluated on the subcases of a case. e.g.{ "subcase_value_source": {"case_property": "name"} "case_type": "child", "is_closed": false, "jsonpath": "$.childrensNames" }
supercase_value_source
: Defines a ValueSource to be evaluated on the parent/host case of a case. e.g.{ "supercase_value_source": {"case_property": "name"} "referenced_type": "mother", "jsonpath": "$.mothersName" }
value
: A constant value. This can be used for exporting a constant, or it can be combined with case_property for importing a constant value to a case property. See corehq.motech.value_source:ConstantValue for more details.
Data Types
Integrating structured data with remote systems can involve converting data from one format or data type to another. Use data type declarations to cast the data type of a value.
For standard OpenMRS properties (person properties, name properties and address properties) MOTECH will set data types correctly, and integrators do not need to worry about them.
But administrators may want a value that is a date in CommCare to a datetime in a remote system, or vice-versa. To convert from one to the other, set data types for value sources.
The default is for both the CommCare data type and the external data type not to be set. e.g.
{
"expectedDeliveryDate": {
"case_property": "edd",
"commcare_data_type": null,
"external_data_type": null
}
}
To set the CommCare data type to a date and the OpenMRS data type to a datetime for example, use the following:
{
"expectedDeliveryDate": {
"case_property": "edd",
"commcare_data_type": "cc_date",
"external_data_type": "omrs_datetime"
}
}
For the complete list of CommCare data types, see MOTECH constants. For the complete list of DHIS2 data types, see DHIS2 constants. For the complete list of OpenMRS data types, see OpenMRS constants.
Import-Only and Export-Only Values
In configurations like OpenMRS Atom feed integration that involve both sending data to OpenMRS and importing data from OpenMRS, sometimes some values should only be imported, or only exported.
Use the direction
property to determine whether a value should only
be exported, only imported, or (the default behaviour) both.
For example, to import a patient value named “hivStatus” as a case
property named “hiv_status” but not export it, use
"direction": "in"
:
{
"hivStatus": {
"case_property": "hiv_status",
"direction": "in"
}
}
To export a form question, for example, but not import it, use
"direction": "out"
:
{
"hivStatus": {
"case_property": "hiv_status",
"direction": "out"
}
}
Omit direction
, or set it to null
, for values that should be
both imported and exported.
Getting Values From JSON Documents
JSONPath has emerged as a standard for navigating JSON documents. It is supported by PostgreSQL, SQL Server, and others. ValueSource uses it to read values from JSON API resources.
And, in the case of FHIR Integration, it also uses it to build FHIR resources.
See the article by Stefan Goessner, who created JSONPath, for more details.
OpenMRS observations and Bahmni diagnoses can be imported as extension cases of CommCare case. This is useful for integrating patient referrals, or managing diagnoses.
Values from the observation or diagnosis can be imported to properties of the extension case. MOTECH needs to traverse the JSON response from the remote system in order to get the right value. Value sources can use JSONPath to do this.
Here is a simplified example of a Bahmni diagnosis to get a feel for JSONPath:
{
"certainty": "CONFIRMED",
"codedAnswer": {
"conceptClass": "Diagnosis",
"mappings": [
{
"code": "T68",
"name": "Hypothermia",
"source": "ICD 10 - WHO"
}
],
"shortName": "Hypothermia",
"uuid": "f7e8da66-f9a7-4463-a8ca-99d8aeec17a0"
},
"creatorName": "Eric Idle",
"diagnosisDateTime": "2019-10-18T16:04:04.000+0530",
"order": "PRIMARY"
}
The JSONPath for “certainty” is simply “certainty”.
The JSONPath for “shortName” is “codedAnswer.shortName”.
The JSONPath for “code” is “codedAnswer.mappings[0].code”.
For more details, see How to Inspect an Observation or a Diagnosis in the documentation for the MOTECH OpenMRS & Bahmni Module.
The value_source Module
- class corehq.motech.value_source.CaseOwnerAncestorLocationField(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, case_owner_ancestor_location_field: str)[source]
A reference to a location metadata value. The location may be the case owner, the case owner’s location, or the first ancestor location of the case owner where the metadata value is set.
e.g.
{ "doc_type": "CaseOwnerAncestorLocationField", "location_field": "openmrs_uuid" }
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, case_owner_ancestor_location_field: str) None
Method generated by attrs for class CaseOwnerAncestorLocationField.
- class corehq.motech.value_source.CaseProperty(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, case_property: str)[source]
A reference to a case property value.
e.g. Get the value of a case property named “dob”:
{ "case_property": "dob" }
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, case_property: str) None
Method generated by attrs for class CaseProperty.
- class corehq.motech.value_source.CasePropertyConstantValue(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, value: str, value_data_type: str = 'cc_text', case_property: str)[source]
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, value: str, value_data_type: str = 'cc_text', case_property: str) None
Method generated by attrs for class CasePropertyConstantValue.
- class corehq.motech.value_source.ConstantValue(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, value: str, value_data_type: str = 'cc_text')[source]
ConstantValue
provides aValueSource
for constant values.value
must be cast asvalue_data_type
.get_value()
returns the value for export. Useexternal_data_type
to cast the export value.get_import_value()
anddeserialize()
return the value for import. Usecommcare_data_type
to cast the import value.>>> one = ConstantValue.wrap({ ... "value": 1, ... "value_data_type": COMMCARE_DATA_TYPE_INTEGER, ... "commcare_data_type": COMMCARE_DATA_TYPE_DECIMAL, ... "external_data_type": COMMCARE_DATA_TYPE_TEXT, ... }) >>> info = CaseTriggerInfo("test-domain", None) >>> one.deserialize("foo") 1.0 >>> one.get_value(info) # Returns '1.0', not '1'. See note below. '1.0'
Note
one.get_value(info)
returns'1.0'
, not'1'
, becauseget_commcare_value()
castsvalue
ascommcare_data_type
first.serialize()
casts it fromcommcare_data_type
toexternal_data_type
.This may seem counter-intuitive, but we do it to preserve the behaviour of
ValueSource.serialize()
.- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, value: str, value_data_type: str = 'cc_text') None
Method generated by attrs for class ConstantValue.
- class corehq.motech.value_source.FormQuestion(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, form_question: str)[source]
A reference to a form question value.
e.g. Get the value of a form question named “bar” in the group “foo”:
{ "form_question": "/data/foo/bar" }
Note
Normal form questions are prefixed with “/data”. Form metadata, like “received_on” and “userID”, are prefixed with “/metadata”.
The following metadata is available:
Name
Description
deviceID
An integer that identifies the user’s device
timeStart
The device time when the user opened the form
timeEnd
The device time when the user completed the form
received_on
The server time when the submission was received
username
The user’s username without domain suffix
userID
A large unique number expressed in hexadecimal
instanceID
A UUID identifying this form submission
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, form_question: str) None
Method generated by attrs for class FormQuestion.
- class corehq.motech.value_source.FormUserAncestorLocationField(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, form_user_ancestor_location_field: str)[source]
A reference to a location metadata value. The location is the form user’s location, or the first ancestor location of the form user where the metadata value is set.
e.g.
{ "doc_type": "FormUserAncestorLocationField", "location_field": "dhis_id" }
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, form_user_ancestor_location_field: str) None
Method generated by attrs for class FormUserAncestorLocationField.
- class corehq.motech.value_source.SubcaseValueSource(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, subcase_value_source: dict, case_types: List[str] | None = None, is_closed: bool | None = None)[source]
A reference to a list of child/extension cases.
Evaluates nested ValueSource config, allowing for recursion.
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, subcase_value_source: dict, case_types: List[str] | None = None, is_closed: bool | None = None) None
Method generated by attrs for class SubcaseValueSource.
- class corehq.motech.value_source.SupercaseValueSource(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, supercase_value_source: dict, identifier: str | None = None, referenced_type: str | None = None, relationship: str | None = None)[source]
A reference to a list of parent/host cases.
Evaluates nested ValueSource config, allowing for recursion.
- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None, supercase_value_source: dict, identifier: str | None = None, referenced_type: str | None = None, relationship: str | None = None) None
Method generated by attrs for class SupercaseValueSource.
- class corehq.motech.value_source.ValueSource(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None)[source]
Subclasses model a reference to a value, like a case property or a form question.
Use the
get_value()
method to fetch the value using the reference, and serialize it, if necessary, for the external system that it is being sent to.- __init__(*, external_data_type: str | None = None, commcare_data_type: str | None = None, direction: str | None = None, value_map: dict | None = None, jsonpath: str | None = None) None
Method generated by attrs for class ValueSource.
- deserialize(external_value: Any) Any [source]
Converts the value’s external data type or format to its data type or format for CommCare, if necessary, otherwise returns the value unchanged.
- get_value(case_trigger_info: CaseTriggerInfo) Any [source]
Returns the value referred to by the ValueSource, serialized for the external system.
- serialize(value: Any) Any [source]
Converts the value’s CommCare data type or format to its data type or format for the external system, if necessary, otherwise returns the value unchanged.
- corehq.motech.value_source.deserialize(value_source_config: JsonDict, external_value: Any) Any [source]
Converts the value’s external data type or format to its data type or format for CommCare, if necessary, otherwise returns the value unchanged.
- corehq.motech.value_source.get_case_location(case)[source]
If the owner of the case is a location, return it. Otherwise return the owner’s primary location. If the case owner does not have a primary location, return None.
- corehq.motech.value_source.get_form_question_values(form_json)[source]
Given form JSON, returns question-value pairs, where questions are formatted “/data/foo/bar”.
e.g. Question “bar” in group “foo” has value “baz”:
>>> get_form_question_values({'form': {'foo': {'bar': 'baz'}}}) {'/data/foo/bar': 'baz'}