This document is meant to orient developers to working with Web Apps. Its primary audience is developers who are familiar with CommCare HQ but not especially familiar with CommCare mobile or formplayer.
High-level pieces of the system:
CloudCare is a legacy name for web apps. Web apps code is in the
cloudcaredjango app. It should not be used in documentation or anything user-facing. It shouldn’t be used in code, either, unless needed for consistency. It mostly shows up in filenames and URLs.
Web apps is tightly coupled with formplayer, so check out the formplayer README.
Is Web Apps Part of HQ? Yes and No.
Web apps is a part of HQ, but once you move into an app, its communication with the rest of HQ is quite limited.
Ways in which web apps is a typical piece of HQ code:
cloudcaredjango app contains the HQ side of web apps.
cloudcare.viewsmodule contains views. Note that there’s just one major view for web apps,
FormplayerMain, and another for app preview,
When you look at the web apps home page, where there’s a tile for each app, those apps come from HQ.
Web apps does have some interactions with HQ once you’re in an app:
The Log In As action works via HQ
HQ provides some system information, like the current user’s username and the mapbox API key, via the original context and initial page data
HQ directly serves multimedia files
Web apps calls HQ analytics code (Google Analytics, Kissmetrics, etc.)
However, in most ways, once you move into an app, web apps only interacts with formplayer and is just a thin UI layer.
Also, before going into an app, on the web apps home page, the sync and saved forms options are formplayer requests.
Example: Case Search
As an example, consider case search, where the user triggers a search that runs against all cases in the domain, not just cases in their casedb, which requires a query to postgres.
If web apps were a typical area of HQ, this might be implemented as a single ajax request:
Instead, formplayer acts as an intermediary. Web apps sends a navigation request to formplayer, which constructs and sends a search request to HQ, which returns a search results response to formplayer, which transforms the results into a case list response and sends that back to web apps.
Note that formplayer does a good deal of processing here. It’s formplayer that determines a search request is needed, and it’s formplayer that processes the results and turns them into a table. Web apps doesn’t even know that a search happened.
This approach clearly isn’t minimizing network requests. However, this architecture is what allows CommCare mobile and web apps to share the majority of their logic, which is huge for developing and maintaining features to work on both platforms.
Anatomy of a Web Apps Feature
The relationships between HQ, formplayer, and mobile mean that web apps work frequently involves working in multiple languages, in multiple repositories, which may have different release processes.
New features require some or all of the following:
App manager UI where the the feature is enabled & configured
App build logic, typically changes to suite generation
New model for the configuration
Formplayer processing to add the new feature to a response
Web apps UI for the feature
CommCare Mobile UI for the new feature
Not all features have all of these pieces:
Some features don’t require any Java
They might use existing flexible configuration, like adding a new appearance attribute value to support a new data entry widget
They might rearrange existing constructs in a new way. CommCare supports a much broader set of functionality than what HQ allows users to configure.
Some features don’t get implemented on mobile.
Some features, like case search, have additional HQ work because they interact with HQ in ways beyond what’s described above.
Example: Registration from Case List
As an example, consider registration from the case list:
A CommCare HQ user goes to the module settings page in app builder and turns on the feature, selecting the registration form they want to be accessible from the case list.
This adds a new attribute to their
Applicationdocument - specifically, it populates
When the user makes a new build of their app, the app building code reads the
Applicationdoc and writes out all of the application files, including the
When a Web Apps user clicks the menu’s name to access the case list, web apps sends a
navigate_menurequest to formplayer that includes a set of
selections(see navigation and replaying of sessions).
The formplayer response tells web apps what kind of sceen to display:
entitieswhich tells web apps to display a case list UI
entitieslist contains the cases and their properties
actionslist includes an action for the case list registration form, which tells web apps to display a button at the bottom of the case list with the given label, that when clicked will add the string
action 0to the
selectionslist and then send formplayer another navigation request, which will cause formplayer to send back a form response for the registration form, which web apps will then display for the user.
Note how generic the concepts web apps deals with are: “entities” can be cases, fixture rows, ledger values, etc. Web apps doesn’t know what cases are, and it doesn’t know the difference between an action that triggers a case list registration form and an action that triggers a case search.
corehq.apps.cloudcare.static.cloudcare.js. As described above, in many ways this code is independent of the
rest of HQ.
Think of the web apps code as split into two major pieces: form entry and everything else.
Form entry contains all interaction while filling out a form: all the different types of questions, the logic for validating answers as the user fills them out, etc. This code is written in a combination of knockout and vanilla JS, and it’s quite old (pre-2014).
Wrapped around the form entry code is everything else, which is controlled by the
application. The single-page application (SPA) approach is unique in HQ.
This is also the only area of HQ that uses Backbone and Marionette.
Most of this code was written, or substantially re-written, around 2016.
In-app navigation, case lists, case search, etc.
Web apps home screen displaying all of a domain’s apps
Log In As
Tight coupling with formplayer means web apps tends to use formplayer/mobile/CommCare vocabulary rather than HQ vocabulary: “entities” instead of “cases”, etc.
The major CommCare/HQ concepts FormplayerFrontend deals with are apps, users, menus, and sessions. “Apps” and “users” are the same concepts they are in the rest of HQ, while a “menu” is a UI concept that covers the main web apps screens, and “sessions” means incomplete forms.
These are HQ apps. Most of the logic around apps has to do with displaying the home screen of web apps, where you see a tiled list of apps along with buttons for sync, settings, etc.
This home screen has access to a subset of data from each app’s couch document, similar but not identical to the “brief apps” used in HQ that are backed by the
applications_brief couch view.
Once you enter an app, web apps no longer has access to this app document. All app functionality in web apps is designed as it is in mobile, with the feature’s configuration encoded in the form XML or suite.xml. That config is then used to generate the web apps UI and to formulate requests to formplayer.
These are HQ users, although the model has very few of the many attributes of CouchUser.
Most of the time you’re only concerned with the current user, who is accessible by requesting
currentUser from the FormplayerFrontEnd’s channel (see below for more on channels).
The users code also deals with the Log In As workflow. Log In As is often described as “restore as” in the code: the user has a
restoreAs attribute with the username of the current Log In As user, the
RestoreAsBanner is the yellow banner up top that shows who you’re logged in as, and the
RestoreAsView is the Log In As screen. The current Log In As user is stored in a cookie so that users do not need to repeat the workflow often.
These are incomplete forms - the same incomplete forms workflow that happens on mobile, but on web apps, incomplete forms are created automatically instead of at the user’s request. When a user is in form entry, web apps creates an incomplete form in the background and stores the current answers frequently so they can be accessed if the user closes their browser window, etc. These expire after a few days, maybe a week, exact lifespan might be configurable by a project setting. They’re accessible from the web apps home screen.
All of this code is stored in
It has top-level directories for the two major areas described above:
form_entry for in-form behavior and
formplayer for the
FormplayerFrontend application. There are also a few top-level directories and files for
The form_entry directory contains the logic for viewing, filling out, and submitting a form.
This is written in knockout, and it’s probably the oldest code in this area.
Major files to be aware of:
Container, the major abstractions used by form definitions.
Containeris the base abstraction for groups and for forms themselves.
Entryand its many subclasses, the widgets for entering data. The class hierarchy of entries has a few levels. There’s generally a class for each question type:
TimeEntry, etc. Appearance attributes can also have their own classes, such as
web_form_session.js defines the interaction for filling out a form. Web apps sends a request to formplayer every time a question is answered, so the session manages a lot of asynchronous requests, using a task queue. The session also handles loading forms, loading incomplete forms, and within-form actions like changing the form’s language.
Form entry has a fair amount of test coverage. There are entry-specific tests and also tests for web_form_session.
The formplayer directory contains logic for selecting an app, navigating through modules, displaying case lists, and almost everything besides filling out a form.
This is written using Backbone and Marionette. Backbone is an MVC framework for writing SPAs, and Marionette is a library to simplify writing Backbone views.
FormplayerFrontend is the “application” in this SPA.
This is everything not in either the
This controls the debugger, the “Data Preview” bar that shows up at the bottom of app preview and web apps and lets the user evaluate XPath and look at the form data and the submission XML.
This contains logic specific to app preview.
There isn’t much here: some initialization code and a plugin that lets you scroll by grabbing and dragging the app preview screen.
The app preview and web apps UIs are largely identical, but a few places do distinguish between them, using the
environment attribute of the current user. Search for the constants
WEB_APPS_ENVIRONMENT for examples.
hq_events.js, although not in this directory, is only really relevant to app preview. It controls the ability to communicate with HQ, which is used for the “phone icons” on app preview: back, refresh, and switching between the standard “phone” mode and the larger “tablet” mode.
This controls the UI for the Web Apps Permissions page, in the Users section of HQ. Web apps permissions are not part of the standard roles and permissions framework. They use their own model, which grants/denies permissions to apps based on user groups.
Inline formplayer is for the legacy “Edit Forms” behavior, which allowed users to edit submitted forms using the web apps UI. This feature has been a deprecation path for quite a while, largely replaced by data corrections. However, there are still a small number of clients using it for workflows that data corrections doesn’t support.
This contains miscellaneous utilities, mostly around error/success/progress messaging:
Error and success message helpers
Progress bar: the thin little sliver at the very top of both web apps and app preview
Error and success messaging for syncing and the “settings” actions: clearing user data and breaking locks
Sending formplayer errors to HQ so they show up in sentry
Code for initializing the markdown renderer including a bunch of code,
injectMarkdownAnchorTransforms and its
helpers, related to some custom feature flags that integrate web apps with external applications.
It’s heavily asynchronous, since it’s a fairly thin UI on top of formplayer. Want to get the a case’s details? Ask formplayer. Want to validate a question? Ask formplayer. Adding functionality? It will very likely require a formplayer PR - see “Anatomy of a Web Apps Feature” above.
doesn’t exactly use globals, but
FormplayerFrontend is basically a god object, and it uses a global message
bus - see “Events” below.
Web apps has only transient data. All persistent data is handled by formplayer and/or HQ. The data that’s specific to web apps consists mostly of user-related settings and is handled by the browser: cookies, local storage, or session storage.
The Log In As user is stored in a cookie. Local storage is used for the user’s display options, which are the settings for language, one question per screen, etc. Session storage is also used to support some location handling and case search workflows.
Note that these methods aren’t appropriate for sensitive data, which includes all project data. This makes it challenging to implement features like saved searches.
For day-to-day web apps development, it’s just useful to know that
FormplayerFrontend controls basically everything, and that the initial hook into its behavior is the
start event, so we have a
before:start handler and a
Marionette’s regions are UI containers, defined in the FormplayerFrontend’s
We rarely touch the region-handling code, which defines the high-level structure of the page: the “main” region, the progress bar, breadcrumbs, and the restore as banner. The persistent case tile also has a region. Most web apps development happens within the
It is sometimes useful to know how the breadcrumbs work. The breadcrumbs are tightly tied to formplayer’s selections-based navigation. See Navigation and replaying of sessions for an overview and examples. The breadcrumbs use this same selections array, which is also an attribute of
CloudcareURL, with one breadcrumb for each selection.
Backbone.Radio and Events
Marionette integrates with Backbone.Radio to support a global message bus.
Although you can namespace channels, web apps uses a single
formplayer channel for all messages, which is accessed using
FormplayerFrontend.getChannel(). You’ll see calls to get the channel and then call
request to get at a variety of global-esque data, especially the current user. All of these requests are handled by
reply callbacks defined in
FormplayerFrontend also supports events, which behave similarly. Events are triggered directly on the
FormplayerFrontend object, which defines
on handlers. We tend to use events for navigation and do namespace some of them with
:, leading to events like
menu:show:detail. Some helper events are not namespaced, such as
Routing, URLs, and Middleware
As in many SPAs, all of web apps’ “URLs” are hash fragments appended to HQ’s main cloudcare URL,
Marionette.AppRouter, which extends Backbone’s router.
Web apps routes are defined in router.js.
Routes outside of an application use human-readable short names. For example:
/a/<DOMAIN>/cloudcare/apps/v2/#appsis the web apps home screen, which lists available apps and actions like sync.
/a/<DOMAIN>/cloudcare/apps/v2/#restore_asis the Log In As screen
Routes inside an application serialize the
CloudcareURL contains the current state of navigation when you’re in an application. It’s basically a js object with getter and setter methods.
Most app-related data that needs to be passed to or from formplayer ends up as an attribute of CloudcareURL. It interfaces almost directly with formplayer, and most of its attributes are properties of formplayer’s SessionNavigationBean.
CloudcareURL is defined in formplayer/utils/utils.js although it probably justifies its own file.
CloudcareURL are not especially human-legible due to JSON serialization, URL encoding, and the obscurity of the attributes. Example URL for form entry:
The router also handles actions that may not sound like traditional navigation in the sense that they don’t change which screen the user is on. This includes actions like pagination or searching within a case list.
Other code generally interacts with the router by triggering an event (see above for more on events). Most of
router.js consists of event handlers that then call the router’s API.
Every call to one of the router’s API functions also runs each piece of web apps middleware, defined in middleware.js. This middleware doesn’t do much, but it’s a useful place for reset-type logic that should be called on each screen change: scrolling to the top of the page, making sure any form is cleared out, etc. It’s also where the “User navigated to…” console log messages come from.
There are tests in the
spec directory. There’s decent test coverage for js-only workflows, but not for HTML interaction.
Web apps development frequently happens in
inherit from Marionette.View. This section
View attributes that web apps most frequently uses.
These attributes link view code with the relevant HTML template.
We typically use
template and just fetch a template by its id, then run it through underscore’s
function. The QueryListView,
which controls the case search screen, is a good example, defining
_.template($("#query-view-list-template").html() || "").
getTemplate is a callback, so it has access to
this and allows for more complex logic. We use it in the
https://github.com/dimagi/commcare-hq/blob/9baa5a05181e3e74cdf8608223eeff69aca5c0d7/corehq/apps/cloudcare/static/cloudcare/js/formplayer/menus/views.js#L35-L43>__ to determine whether to display the menu in a list style or in a grid style.
All views have a single encompassing container, which is added by Marionette, so it doesn’t show up in the view’s HTML template. These attributes influence that container.
tagName, which can be a string or a callback, defines the HTML node type, typically
className allows setting a CSS class on the container.
attributes allows setting HTML attributes. We mostly use this for accessibility, to set attributes like
initialize is for any setup, particularly for storing any options that were passed into the view
this.options is available throughout the view).
templateContext is for building an object of context to pass to the template, as with
_.template and django
onRender is called every time Marionette renders the view. We use this primarily for attaching events to
content. Note that Marionette has its own attributes for event handling, discussed below, but
useful for non-standard events provided by third-party widgets like select2 and jQuery UI.
These attributes are for event handling.
ui is an object where keys are identifiers and values are jQuery selectors. Elements defined in
available to other code in the view using
this.ui. For an example, see how QueryListView
defines ui elements for the case search screen’s submit and clear buttons.
events ties elements from
ui with standard HTML events. Events references the event, the ui element, and
the callback to invoke. Again, QueryListView
is a good example.
modelEvents attaches callbacks to events on the Backbone model, as opposed to ui events. We don’t use this
often, but QueryView,
which controls an individual search field on the case search screen, uses it to force the view to re-render
whenever the underlying model changes, so that select2 behaves properly.
These options apply to views that extend Marionette.CollectionView. These views are structured to display a list of child views. As an example, QueryListView controls the case search screen and has a child QueryView for each individual search field. The case list’s CaseListView is a more complex example, with a CaseView child view that has several subclasses.
childView names the view that is a child of this view.
childViewContainer tells Marionette where in the parent view to render the children. This can be an HTML node
name, analagous to
tagName, or it can be a jQuery selector identifying a specific element in the view that
should contain the children.
childViewOptions allows the parent view to pass data to the children views. Some use cases: