Does Information and Communication Technology Add Value to Citizen-Led Accountability Initiatives in Health? Experiences from India and Guatemala

Marta Schaaf, Shruti Chhabra, Walter Flores, Francesa Feruglio, Jashodhara Dasgupta, and Ana Lorena Ruano

Abstract

Information and Communication Technology (ICT) may facilitate the collection and dissemination of citizen-generated data to enhance governmental accountability for the fulfillment of the right to health. The aim of this multiple case study research was to distill considerations related to the implementation of ICT and health accountability projects, describe the added operational value of ICT tools (as compared to similar projects that do not use ICT), and make preliminary statements regarding government responsiveness to accountability demands through ICT projects. In all three projects, the need for relationship building, continuous community engagement and technical support, and training for volunteers or service users was identified. Government responsiveness to the data varied, suggesting that political will is lacking in certain contexts. Despite these challenges, ICT initiatives provided an easy, accessible, and low-risk platform for reporting violations and demanding accountability from service providers and decision-makers. ICT-enabled citizen generated data can add significant operational value and some political value to project activities and goals, and may affect systems change when it is part of a broad-based, multi-level civil societal and governmental effort to improve health care quality.

Introduction

Global and national health policy-makers and citizen activists agree that accurate data on health service coverage, equity, and quality is a vital public good that should inform health policy-making and health governance.¹ This assertion is echoed in human rights norms related to the right to information, the right to demand compliance and answerability from state actors, and the right to see one’s priorities reflected in health policies.² However, it is possible that in some contexts, health providers, administrators, and politicians resist the aggregation, dissemination, and utilization of data because it could expose corruption, under-investment, and implementation failures.³ Non-governmental organizations (NGOs) and activists may compile their own data on user experiences of health care, with a view to exposing critical gaps in government fulfillment of the right to available, accessible, acceptable, and good quality health services (hereinafter AAAQ).⁴ As documented in the health and human rights field and in the growing field of social accountability, successful efforts to collect and leverage such data have led to improvements in various AAAQ attributes of health care, including respectful treatment and decreased absenteeism. Some efforts have set the stage for continued improvements by fostering individual and collective empowerment; and sustained changes in relations between marginalized communities and the government.⁵

Many donors and advocates are optimistic about the role information and communication technology (ICT) might play in the collection and dissemination of citizen-generated data to enhance governmental accountability for health care services.⁶

The consensus definition of accountability encompasses answerability and enforceability. Governments are obligated to provide information and justification regarding policy and practice (answerability), and sanctions must exist for failure to comport with policy (enforceability).⁷ Human rights actors stress the remedy component of enforceability, including, for example, restitution, compensation, rehabilitation, satisfaction, and guarantees of non-repetition.⁸ In this paper, we adopt a right to health framework: governments should be accountable for the progressive realization of the right to health, including delivering AAAQ health services.⁹

The emerging grey and peer-reviewed literature suggests that while ICT can be an effective tool for aggregating data to demand accountability, unbridled optimism regarding ICT for health accountability is not warranted.¹⁰Collecting and disseminating data is only one step in fostering state accountability; in order for government services to improve, the state must take concerted action and must have the capacity and willingness to do so in effective ways.¹¹ Nonetheless, ICT may be useful to citizen monitoring of public service delivery in specific settings. It could make data gathering and aggregation easier, thus adding operational value, and could augment the political cost and the perceived legitimacy of claims made by citizens, thus adding political value.

The comparative case study described here emerged from discussions within the Community of Practitioners on Accountability and Social Action in Health (COPASAH), a network of practitioners and organizations working in low- and middle-income countries. COPASAH fosters learning exchanges and conducts joint advocacy. This study was designed by practitioners from three COPASAH member organizations: Centro de Estudios para la Equidad y Gobernanza en los Sistemas de Salud (CEGSS), Nazdeek, and SAHAYOG, with support from researchers at the Mailman School of Public Health at Columbia University.

Comprehensively assessing how ICT contributes to specific outcomes relating to the right to health is a long-term goal. Our research addressed the more intermediate goal of identifying and comparing operational questions that will shape the extent to which ICT adds value to efforts to enhance government accountability for health.

The aim of this paper is to describe lessons learned emerging from the comparative case study, including:

considerations related to project implementation;
the added operational value of ICT tools (including websites and mobile phones; as compared to similar projects that do not use ICT);
challenges and adaptive program changes; and,
government responsiveness to accountability demands through ICT projects.

We intend this paper to be a practitioner-led contribution to the growing peer-reviewed literature regarding citizen-led accountability and ICT. Most notably, between 2012 and 2017, Making All Voices Count (MAVC), an international funding and research consortium, supported an in-depth multi-country study specific to ICT for improving public service accountability.¹² Our research is unique in that we focus exclusively on the availability and quality of health services for low caste, tribal, and indigenous populations. These groups are among those most likely to experience non-AAAQ health care.¹³ They are also potentially less likely to a) have information about their entitlements, b) view themselves as rights holders, and c) be technologically literate and therefore able to use ICT tools effectively to lodge complaints.¹⁴ They are disproportionately removed from the human rights tools enabling them to report and seek remedy for violations of the right to health.

Methods

Study context

In the three projects considered in this paper, project volunteers or health service users from the community employed mobile phone technology (interactive voice response and/or text messages) to gather or report evidence from government health facilities regarding a pre-determined set of indicators of quality health care. These indicators were not meant to capture the entire concept of quality, but to portray attributes of care that communities had identified as important and that lay people could assess, such as disrespectful treatment and demands that patients make informal payments for health care. The data were then aggregated, mapped, and displayed through online platforms developed by the organizations. The projects all used the same basic theory of change: the implementing NGOs and the communities they serve would define their health care priorities, learn about relevant legal entitlements and standards, use ICT to document users’ experiences against the standards, and deploy the data generated to demand government response. Consistent with the human rights principle of remedy, Nazdeek and CEGSS also sought restitution for aggrieved patients in some cases. All three organizations sought satisfaction and guarantee of non-repetition, such as public disclosure of the situation, acknowledgement of problems, or a change in legislation or policy.

The implementing organizations created opportunities for users to engage with the government health system collectively at multiple levels in order to demand that they address the identified problems. The implementing NGO complemented this collective engagement with ongoing advocacy such as one-on-one meetings with district health officials. The NGOs expected the health system to respond for two key reasons: (1) health facility leadership and other decision-makers would learn more about current challenges in health care quality, including the frequency of certain challenges and how they affected the community, and (2) health providers and leaders would be politically or professionally compelled to act because publicly displayed data regarding poor implementation of service delivery would engage the public, heighten political costs, and/or alert the formal health system hierarchy about problems. Though the basic theories of change were similar, the program activities differed in several key ways, reflecting the different orientations and organizational goals of the NGOs, as well as the very different contexts in which the efforts were undertaken.

CEGSS

CEGSS was formed in 2006 with an aim to use participatory approaches to improve indigenous Guatemalans’ access to quality health care, particularly care that was culturally acceptable, non-discriminatory, and reflected community priorities. The project described in this study has been operational and evolving for 10 years; CEGSS changed it from a paper-based system to an ICT-enabled system in 2014. As part of this program, CEGSS trained and supported indigenous Community Defenders for the Right to Health (CDRH) who visited health facilities regularly. During these visits, CDRH asked users if they had faced any obstacles to obtaining care. They also interviewed front-line health care providers and inspected facilities. CDRH conducted health rights awareness-raising campaigns in communities and encouraged community members to relay challenges they faced in accessing quality care. They reported identified problems on an SMS platform. These complaints were then mapped on Ushahidi, an open source monitoring platform that was initially developed to monitor election violence in Kenya (Figure 1).

Nazdeek

Nazdeek, a legal empowerment organization, was formed in 2012. As part of their End Maternal Mortality (MM) Now Project, Adivasi (tribal) women volunteers identified and reported cases of maternal and infant health care violations occurring in tea plantation areas in Sonitpur District, Assam, India. Women could report violations via SMS, using numeric codes corresponding to specific locations and types of violations. Like the CEGSS project, the cases were mapped onto a publicly accessible Ushahidi platform (endmmnow.org). Locations included facilities run by tea plantation managers, which are legally required to provide basic health care to their workers; as well as public facilities, such as hospitals; Anganwadi centers (community centers providing health and food services to women and children); and ration shops. To implement the project, Nazdeek worked with a local organization and an international partner.

SAHAYOG

SAHAYOG, a women’s health and rights organization based in Uttar Pradesh (UP), India, was established in 1992. SAHAYOG has a long-standing collaboration with community-based organizations (CBOs) throughout UP and with a grassroots women’s forum (Mahila Swasthya Adhikar Manch or MSAM), that is comprised of approximately 12,000 poor, rural women leaders from Dalit, Muslim, and tribal communities. In 2011, the government initiated a scheme to ensure free comprehensive maternity care, and the UP state government requested that civil society organizations monitor its implementation independently. In 2012, SAHAYOG launched a project to use interactive voice response to document health provider demands that women make informal payments for maternal health care services they were entitled to receive free of charge. A two-district pilot version of the project was evaluated in 2013, and then the project was refined and scaled up to four districts in 2014.¹³These reports were also mapped in real time via Ushahidi on a project website.

Each of these organizations conduct additional local and national level activities beyond the projects described here. While we do not discuss these other activities in this paper, it is important to acknowledge that the ICT projects are implemented in the context of ongoing engagement with the communities and the local and national governments addressed by the ICT efforts.

Research approach

We used a comparative qualitative case study approach to examine these three projects, which allowed us to consider the contextual particularities of each project, implementing organization, and site. While the extent of the current research was somewhat limited by budget, we sought to generate practical, action-oriented, context-dependent knowledge that is useful to practitioners.¹⁵

The research reported here represents an effort to go beyond routine monitoring and evaluation to facilitate our functioning as “learning organizations,” to distill operational lessons about the use of ICT for the benefit of our own programs and beyond. We have worked on these issues for many years. We do not expect that there is a “magic bullet” project that can fix the problems we address. Instead, we try different strategies and assess whether or not these strategies—or elements of them—seem to work better than previous efforts.

Data collection

Table 1 summarizes the methods for each single case study. All three studies used semi-structured in-depth interview (IDI) and focus group discussion (FGD) guides for discussions with implementers. Researchers from all sites developed individualized research tools using a jointly agreed list of themes that were articulated based on a literature review, an initial in-person meeting regarding our programs and current knowledge, policy discussions on ICT and development, and priority questions and challenges we faced in implementing the ICT efforts. We were interested in factors related to program design, implementation, impact, and context, such as anonymity of the complainant, engagement from local level health providers, and health planner utilization of the data generated through the project. Our questions (and findings) relate primarily to the process of program implementation, with some insight regarding government responsiveness to the human rights claims. Comprehensively assessing changes in the right to health as a result of local level social accountability projects such as ours is difficult; we do not expect that aggregate health indicators will be affected given the fact that our projects are locally bound and implemented in a somewhat limited time frame.

In addition to the IDIs and FGDs with implementers, each project team developed project-specific data collection methods, such as review of project reports and interviews with additional stakeholders.

Table 1. Methods

	Study site and justification	Language and data analysis	Data collection methods
CEGSS	Study conducted in seven municipalities within Huehuetenango, Totonicapan, Alta Verapaz, Solola, and Quiche provinces.	● IDIs and FGDs conducted using semi-structured guides. ● Data collected in Spanish. ● Analysis and synthesis done by CEGSS staff and written up in English.	● Quantitative data analysis of 228 complaints sent to the platform between August 2014 and March 2015 ● Review of monitoring and evaluation reports ● Eighteen IDIs with community members and leaders
Nazdeek: End MM Now	Study conducted in two blocks of Sonitpur District in Assam: Balipara and Dhekiajuili.	● Data collected using semi-structured guides. ● Conducted in local language, transcribed, and translated into English on site. ● Analysis done by researchers at Columbia University, with regular discussions and checks with the field-based team.	● Data collected as part of routine monitoring and assessment of the End MM Now project, including quantitative assessment of 268 reports made by women volunteers from January 1, 2014 to September 30, 2015. ● Review of internal reports. ● IDIs with block administrative officials (three) in governmental facilities and frontline staff (three) Accredited Social Health Activists. ● IDIs with one staff member from Nazdeek, one staff member from Pajhra (partner organization), two field coordinators (women volunteers). ● One FGD with women volunteers in each of two blocks.
SAHAYOG: My Health, My Voice	Two of the project’s four districts, Azamgarh and Mirzapur, selected based on high rates of reporting over time, anecdotal reports of change, and the long-term presence of the women’s collective Mahila Swasthya Adhikar Manch (MSAM). Relationship of MSAM with the government varied over the two districts, providing an opportunity for contrast	● IDIs and FGDs conducted in Hindi, voice recorded, transcribed, and translated to English. ● Analysis done by researchers from Columbia University, one of whom did much of the data collection in Hindi, with input and checking from SAHAYOG. ● Ethical approval from Columbia University and the Sigma IRB in UP.	● Embedded in a larger research project. ● Key informant interviews with seven district officials and health providers, four MSAM members, four SAHAYOG personnel, and four CBO representatives. ● Eight FGDs with MSAM members in each of two districts, with a total of 52 participants; two FGDs with CBO partners with a total of seven participants. ● Nine days of non-participant observation in four facilities, including informal interviews with health care administrators and patients. ● Quantitative analysis of 1,876 calls made to the hotline between January 1, 2014 and September 30, 2015.

Data analysis

Two researchers at Columbia (SC and MS) read all the Nazdeek and SAHAYOG transcripts and an English language data synthesis from CEGSS and, using the initial list of themes that emerged from the literature review and in-person meeting, proposed a shortened list of relevant research themes. Themes included how the ICT mechanism compared with other governmental and non-governmental modes of filing complaints, differences in the design of the ICT tools (such as whether or not anonymity is offered and if the complaint is verified), usability of the ICT tool, ways in which various stakeholders perceive the ICT tool as being different (better or worse) than pen and paper approaches from both a process and an outcome perspective, and risk for those making complaints. These themes were then entered into an Excel sheet, followed by data from each of the three cases. This sheet was shared with all authors for feedback, helping to ensure consistency in data analysis. Entering data into an Excel spreadsheet facilitated triangulation of data collected via different methods, and displayed discrepant data. Thus, we matched observed patterns in the data to our research themes, allowing us to “see” the data supporting various propositions. We found that all three projects made programmatic changes through the implementation. Since our data display gave a cross-sectional view of each project, we went back to the transcripts and wrote brief memos summarizing the program changes and the justification for each.

The Columbia researchers then drafted the paper and shared it with researchers from all sites for significant input through seven rounds of review.

Findings and discussion

We group our findings into four categories: 1) project implementation, 2) added operational value of ICT, 3) challenges and adaptive program changes, and 4) government responsiveness to right to health accountability demands made through ICT projects. While our research focused on ICT, we do not limit our discussion to ICT; offline activities were an integral part of the projects’ theory of change.

Project implementation

Table 2 summarizes key descriptive findings related to project implementation. Please note that the offenses are all alleged; very few cases resulted in a formal legal process to eventually determine culpability.

Table 2. Key descriptive findings

Organization	Who files the complaint	Complaint verification process	Degree of anonymity	Complaint trajectory	Issues raised
CEGSS	Trained community defenders for the right to health (CDRHs; volunteers).	CEGSS staff member calls CDRH to ensure that complaint was categorized properly.	Complainant name is recorded but not put on website. Complainant information is used in the event that CEGSS and complainant wish to pursue legal action. Offending provider is recorded but not displayed on website.	Complaints mapped on website are used for advocacy purposes. Complaints are aggregated by type and formally transmitted to the relevant governmental entity. CEGSS supports complainants who wish to also make a formal complaint to the relevant governmental entity.	● Informal payments. ● Disrespectful / rude treatment. ● Lack of supplies. ● Absenteeism. ● Denial of care. ● Undue referrals. ● Poor infrastructure.
Nazdeek	Trained Adivasi (indigenous) women volunteers or CBO members.	Nazdeek staff member calls complainant to verify details of incident.	Complainant name is recorded for verification purposes but not put on website. Offending provider name is not collected – however in the case of small facilities such as ration shops and Anganwadi centers, the provider is easily identifiable.	Complaints are mapped on the website, aggregated, and submitted to local authorities during Community Grievance Forums with women volunteers and local health officials, facilitated by Nazdeek. In many cases, Nazdeek works with volunteers to file administrative complaints through existing grievance mechanisms or Right to Information requests.	● Informal payments. ● Lack of supplies and equipment. ● Lack of health staff. ● Denial of care, abuse, and discrimination on grounds of gender, ethnicity, and religion. ● Undue referrals. ● Poor infrastructure and hygiene.
SAHAYOG	Trained CBO member or the complainant.	SAHAYOG verifies 10% of calls as a quality check.	Neither the complainant nor the offending provider are named or recorded.	All complaints are mapped online, and data regarding type of complaint and amount of money are posted on the website. This data is used in subsequent advocacy. Patients facing denial of care can call an emergency hotline that is always staffed.	● Informal payments, which at times result in denial of care if patients refuse to pay.

Each project raises similar issues, though the relative frequency of each type of complaint varied.

The projects included multiple “offline” activities to encourage community members to make reports or to raise concerns, and to spur governmental response to the data collected. These activities included

strengthening grassroots associations,
raising public awareness about health-related entitlements and about the project,
NGO leadership meeting health facility leadership and staff,
holding community assemblies or dialogue meetings with structured opportunities for citizens and health providers and decision-makers to interact, and
engaging with authorities at multiple levels of the health system (for example, health facility, district, regional, national) to present the data and advocate for action.

When interviewed, organizational leaders and program implementers made two key points about project implementation, as related to ICT.

Laying the groundwork

As identified in other studies, realizing the potential operational efficiencies associated with ICT requires that the ICT is adopted and used correctly by as many people as possible.¹⁶ Program implementers explained that prolonged, intensive engagement with the community, local organizations, and governmental actors was key to laying the groundwork for implementation. This preparatory engagement included educating stakeholders about the rights and entitlements in question, the intent of the program, what the data could reveal and how to use it, and program operations, including how to use the ICT tools. SAHAYOG’s and Nazdeek’s project relied on women from the community making reports, adding the imperative for the NGOs to learn about community member ability to use the required ICT tools and to build their capacity to do so.

Education and support were essential, as the women making reports were generally low-caste or tribal, and often illiterate. SAHAYOG’s project included an explicit conscientization component, creating conditions for the women to learn about human rights in order to understand that these rights apply to them so that they were motivated to make reports.

Pre-existing relationships in the community and with the government

Existing relationships with concerned community members made ICT uptake easier and strengthened community support for the project. All of the organizations had pre-existing relationships established over the course of past work, and felt they had a base of trust in most of the communities addressed. This facilitated local NGO and community willingness and motivation to participate in the project as well as to use ICT, which was somewhat novel for many.

Moreover, in the case of all three organizations, the projects described here entailed introducing ICT to ongoing initiatives. To varying degrees, communities were already trained and engaged in health facility monitoring, and local health facilities were aware of these efforts.

Data from SAHAYOG demonstrates the importance of this prior engagement. In addition to working with CBOs and MSAM in communities where they had worked for many years, SAHAYOG entered some new communities for this project. The reports were much less frequent in the new SAHAYOG communities, suggesting the facilitative role played by a long-term presence.

Added operational value of ICT

Interviewees reported that ICT enhanced the projects in several ways. These included:

Efficiently collected contemporaneous data in easy-to-understand formats. Data recorded on ICT can be available immediately to the government, citizens, advocates, and the media. As described by CEGSS, this enables faster and more effective collaboration with the media. CEGSS representatives referred journalists to real-time online data, rather than waiting for CDRHs to gather and collate data over a number of months. For instance, professional investigative journalists confirmed the authenticity of complaints regarding demands for informal payments from patients requiring ambulance transport, and then published a report on corruption and the poor state of emergency care facing rural populations.¹⁷

Moreover, all three implementing NGOs suggested that the up-to-date data could inform governmental decision-making. Making extensive field visits in order to collect and aggregate data by hand is time consuming and resource intensive, such that the data available may not be current. In contrast, data collected and reported immediately by ICT can be considered almost real-time. In Guatemala, recent SMS complaints and photos of empty medical shelves at rural facilities were quickly presented to the Minister of Health as evidence that medicines had not yet reached rural areas. As a result, the minister ordered an investigation about why medicines were stored in regional warehouses and not distributed to rural facilities.

Accessible database of complaints and violations that allows easily identifying and visualizing gaps. The efficiency in data collection allowed the organizations to gather and easily manipulate larger amounts of data, which was then aggregated and mapped on the project websites. These presented the data in a way that clearly indicated the scope of certain health sector problems, showed trends, and surfaced “hotspot” facilities with particular challenges.

Analysis of the data allowed the implementing organizations to capture a fuller picture of the health infrastructure in areas that are difficult to reach, such as especially remote tea gardens in Assam and, over time, to identify patterns of violations, such as systemic flaws (for example, scarcity of blood supplies or a flawed referral system) or recurring issues in a given facility (for example, cases of informal payments linked to health workers in a certain hospital). For example, the data in Uttar Pradesh showed that about one-fifth of the cases of informal payments were cases where patients were instructed to buy drugs or other supplies at a private pharmacy outside the health facility, even though patients were entitled to free supplies in the health facility. These types of data helped to guide the organizations’ advocacy efforts, as they could prioritize certain issues and/or areas and make more informed demands to health authorities. In the SAHAYOG example above, as a result of the data collected, SAHAYOG decided to try to better understand the relationship between private pharmacies and health providers. They wanted to ascertain whether genuine drug and supply stock outs or corruption (or both) drive provider requests that patients purchase drugs from the outside, and how government might intervene to fix the problem.

This finding is in line with existing human rights and public health literature on the importance of data for illustrating systemic flaws, thus strengthening the value of ICT platforms in public policy debate.¹⁸Such data could be compiled and analyzed using paper, but the use of ICT makes such analysis easier to conduct and to communicate visually.

In the case of CEGSS and Nazdeek, the databases also served as a case management tool for individual case resolution. Nazdeek’s internal database tracks the actions taken to address some of the cases (such as administrative complaints, requests under the Right to Information Act, and representations with authorities) and their outcome. In the case of CEGSS, the online platform is color coded to track how each complaint moves through the system from the time it is received (red), investigated by a government official (yellow), and resolved (green).

Accessible database of complaints and violations that allows easy identification and visualization of gaps. In the three project sites, the government also maintained its own health sector telephone complaint system, as well as an official system for written complaints for at least some of the period during which the ICT platforms were active. The organizations hoped that their ICT-based monitoring would be easier to use, less risky, and more comprehensive than the government systems in addressing complaints and improving health service delivery.

Community members and NGO staff explained that marginalized communities generally preferred making complaints to the civil society run ICT reporting systems rather than making official, in-person complaints. First, in most cases, the projects ensured anonymity of the complainants, protecting them from retaliation. Second, ICT allows users of services to make complaints without an interaction with an intimidating bureaucrat. Third, no travel is required. Travel can be a real obstacle for poor women or any other user with significant family and work responsibilities in areas that are poorly served by public transportation, such as Assam’s tea gardens, and rural Guatemala and Uttar Pradesh.¹⁹

It is possible that in-person visits from discreet NGO staff would also meet client preferences for convenient, confidential reporting, but again, such a human resource-intense approach to reports seems infeasible in the long run, particularly on the scale of the SAHAYOG project. In fact, it would be difficult for Nazdeek and SAHAYOG to collect data themselves. In the case of Nazdeek, the tea plantations where the project took place were quite remote; tea plantation managers sought to limit civil society groups’ access to the plantations; and, plantation residents seen talking to Nazdeek could be at risk for retaliation. Thus, complainants are far more likely to use a system managed by women living in plantation areas.

The situation was similar for SAHAYOG. While the use of ICT required significant support initially, it facilitated citizens making their own complaints, as opposed to NGO staff going to each site to collect reports. This likely lessened the risk of retaliation, since patients were not seen interacting with CBO staff. Moreover, the SAHAYOG project was implemented on a larger scale than the other two. Though the use of ICT required significant support to communities, it also facilitated scale, since most complaints could eventually be made without SAHAYOG or CBO involvement.

Challenges and adaptive program changes

Each project encountered multiple difficulties, some of which were similar across projects. Implementers altered the programs in response to some of these challenges. We summarize the commonalities and differences in challenges faced across programs. All of these challenges illustrate the fact that the gains in efficiency offered by ICT require significant financial resources, on the ground support, ICT expertise within the implementing organization, and concerted efforts to make ICT accessible to marginalized populations.

Limited technological literacy. As noted, the projects target poor and otherwise disenfranchised communities that may lack rights awareness and technological know-how and confidence to participate in the project. This problem was particularly acute for SAHAYOG, which relied on the aggrieved women, rather than trained advocates or volunteers, for reporting. SAHAYOG staff and partners reported that women required significantly more support to make reports via interactive voice response than initially anticipated. In response, SAHAYOG simplified the interactive voice response system, making it easier for women to select the category of informal payments demands that fit their situation.

Women in Uttar Pradesh also indicated that they faced problems posed by poor phone networks, lack of access to electricity (to charge the phone), and gender norms around phone use. In many households, the husband controlled the family mobile phone.

Still, the need for support decreased over time in Uttar Pradesh. Concern about support for the use of ICT is less relevant in contexts where trained volunteers make reports, but even so, they often require significant training and ongoing support. Nazdeek relied on women volunteers in tea garden areas, who, though trained on health entitlements and the use of the mobile phone, were not familiar with technology and had low literacy levels. Some volunteers reported orally to other volunteers or called the coordinator to make reports (a volunteer coordinator was assigned to each block). Nazdeek sought to address these challenges by increasing the number of in-person meetings with volunteers and later with the community at large.

CEGSS endeavored to minimize the problem posed by low tech literacy by giving communities a role in the design of the technology. Community leaders in Guatemala were involved in the entire process of designing the complaints platform—from the forms to the project website. The developer teams also field-tested the data collection forms in three different sites. However, CEGSS noted that, as found in many other human rights efforts, the poorest and most marginalized were often not present and/or able to take part in community participation fora, including those regarding the design of the project.²⁰

ICT capacity required within implementing organization. ICT capacity was also required within the organization running the project. None of the organizations had implemented such ICT-dependent projects in the past; they were initially unaware of the extent of financial and human support required to keep the programs running smoothly. They sought unexpected technical help, which had financial implications. They all reported glitches in implementation and suggested that other organizations seeking to implement similar projects should carefully outline and consider the time and money required, as well as have realistic expectations about the role of technology in their theories of change.

Adequate technical and program training, support, and feedback to ensure high levels of reporting. The volume of reporting ebbed and flowed. In addition to the challenges described above, all three projects encountered misunderstanding among project staff/volunteers, who would report problems of the same type only once. For example, a health defender in Guatemala would report one case of a medicine stock out at a facility, and then fail to report any subsequent instances of patients finding this medicine out of stock, since it had already been reported. This undercut a key objective of the projects, which was to show how many people were affected by health service problems, highlighting patterns of health entitlements failures. Additional training and support in all projects remedied this problem.

Empathy for the providers’ plight, and the desire to prevent conflicts between frontline health providers and users arising from complaining, were other factors limiting reporting. For example, some community members in Uttar Pradesh explained that they were reluctant to report because they knew that health providers have few resources at their disposal. They felt that health facility infrastructure needed to be upgraded before complaining could be justified. To counter this concern, the three organizations emphasized that data were being collected to facilitate system-level changes, rather than to hold individuals in health facilities responsible, except in cases of egregious rights violations.

Community and volunteer misperceptions that the project was going to have an immediate impact also shaped reporting rates for both Nazdeek and SAHAYOG. Nazdeek struggled to maintain commitment among CBO volunteer reporters. In some cases, volunteers who did not see immediate, tangible changes in response to their work lost interest and slowed or stopped reporting. Women who initially committed to participating in the Nazdeek program did not feel motivated to continue participating after they had collected data. Yet, the next stage was an essential component of the program, where the data were being discussed with health authorities.

The three organizations tested different modes of communication and publicity to maintain community and volunteer engagement. SAHAYOG used stickers, pamphlets, graffiti (wall writing), and street theater, and, insofar as possible, engaged the local community health workers. In response to decreases in reporting, they also began to work with local CBOs to provide updates and feedback to communities on the progress of the project, explaining how the project reports and data were used to advocate for change.

CEGSS made radio announcements, distributed flyers, and organized meetings of community defenders at least once every three months to share their experiences, jointly troubleshoot problems, and encourage the defenders to educate their respective communities about the project.

Nazdeek held regular meetings with volunteers and conducted a participatory midterm evaluation of the project. As a result of volunteers’ requests made during the evaluation, Nazdeek planned additional workshops and community meetings. They also sought to ensure that volunteers were recognized for their work by holding community meetings and providing volunteers with project identification cards. Over time, the number of volunteers participating in the project was reduced, but those who remained involved gained leadership in their communities and were increasingly keen to raise issues with respective authorities, even outside the grievance redressal forums.

While the need for technical support is unique to ICT, the challenge of providing feedback to community members and volunteers making reports as part of health system monitoring projects is not. However, the feedback challenge may be especially important to consider in ICT projects aiming to minimize program costs and operate at significant scale. While ICT seemingly decreases the need for human resources to implement the project, it does not obviate the need for feedback to communities. Implementers might consider using the ICT platform to provide feedback about how individual complaints are contributing to systems data, and how these systems data are being leveraged to demand answerability and remedy.

Risk. Women and CBO personnel in all three sites faced some level of risk for reporting or addressing shortcomings in the health system. CEGSS explained that some community members who have filed reports via official governmental channels experienced intimidation, assaults, and death threats against themselves and their families from public officials and employees. Representatives from Nazdeek and SAHAYOG explained that if an Accredited Social Health Activist, or ASHA (a cadre of community health worker in India), brings a patient to a health facility and the patient refuses to make an informal payment, the next patient the ASHA brings may be denied care. Nazdeek volunteers working in tea gardens were afraid of reporting about malpractices in tea garden hospitals due to risk of retaliation from their employer.

Because of these risks, the three projects offered anonymity to anyone making reports. Since the projects intended to produce aggregated data illustrating patterns of experiences with the healthcare system, patient names were not required. However, this anonymity may weaken some health system responsiveness and redress. For instance, when Nazdeek staff submitted data regarding a number of cases of informal payments in one of the Primary Health Centers, district authorities requested the names of the concerned women in order to intervene. Nazdeek instead disclosed the name of the doctor charging money and the amount paid in exchange for the services provided. In the case of SAHAYOG, several health facility managers interviewed in Uttar Pradesh claimed that they could not address informal payments unless they had the name of the complainant.

NGO staff explained that they needed to navigate these tensions among risk to the client, client desire for individual remedy, and governmental requirements for pursuing official complaints. All three organizations kept the names of complainants confidential (SAHAYOG did not even collect names) and emphasized the importance of producing data that could be used to secure policy changes and governmental guarantees of non-repetition. However, both CEGSS and Nazdeek had channels for individual clients to pursue cases to demand restitution, compensation, or rehabilitation.

Government responsiveness: progress toward fulfillment of the right to health

This study did not entail an exhaustive exploration of government responsiveness to the ICT for health accountability projects. However, improvements in government fulfillment of the right to health, as it related to ICT, were discussed in interviews and focus groups.

As noted, the SMS-based reporting allowed the three organizations to ascertain patterns in rights violations, to collect data on a large scale over a sustained period of time, to make that data available in real time, and to display the data in user-friendly formats. Between May 2014 and September 2017, 268 records of maternal rights violations were recorded and mapped on the End MM Now website. Between January 1, 2014 and September 30, 2015, there were 7,159 violations recorded through 1,876 calls to SAHAYOG’s My Health, My Voice hotline. And between August 2014 and March 2015, 228 violations were reported to CEGSS.

The gathering and deployment of large-scale data likely resulted in a somewhat better response from the government. In the case of SAHAYOG, officials at all levels discussed the data with SAHAYOG and CBO partners, and some high-level officials committed to addressing the problems identified. The state government issued an order that hospital committees include a review of the complaints registered in the hotline in their regular meeting agenda; this order could be implemented because the committees could easily access the data online. Some hospitals made changes; for example, one primary health care facility responded to the evidence presented by establishing a mechanism in partnership with a private clinic to ensure the availability of drugs free of charge. Earlier research on the SAHAYOG project also found that the “elite cachet” associated with ICT seemingly bolstered officials’ willingness to engage and believe the data.²¹

However, despite the SAHAYOG data showing patterns of service gaps, many within the government addressed the problems raised as being attributable to a few miscreant providers, rather than to systemic weaknesses in the health system. They did not seem to integrate the data provided by the program into their decision-making routines; they generally consulted the data only when it was shown to them at meetings and dialogues. For the especially intransigent issue of informal payments, ICT data affected decision-makers in some contexts but not others. As described by SAHAYOG and CBO staff and health facility managers, the key factors seemed to be the professional commitment of the particular health facility manager, and the breadth and depth of the “corruption nexus” in the health facility at issue.

CEGSS had similar experiences with the aggregated data they presented to governmental agencies. Representatives of these agencies were willing to meet and discuss the data in open community dialogues. CEGSS was somewhat successful at instigating facility level changes, with reductions in absenteeism and apologies from providers following reporting. In isolated cases, partners and allies outside of CEGSS used the ICT-gathered data to successfully push for change. For example, in the Solola region, delegates of the San Pablo la Laguna Community Council presented evidence from the platform at Municipal Development Council meetings to describe the challenges people faced in accessing quality health care. In response, the council worked with health center staff to subsidize medicine for the poor, provide subsidies for maintenance of the ambulances and for patient transfers, and finance the improvement of surgical and medical equipment and the construction of a new health post. There were similar outcomes in other municipalities.

In the case of Nazdeek, local authorities demonstrated openness to receive the data. In interviews, they stated that they perceived the data to be reliable. There were anecdotal reports of resultant changes in health service delivery, such as decreases in informal payments, regular disbursement of cash entitlements, and improvements in the condition of the local district-level hospital. Nazdeek staff reported that the data collected through ICT also led to the opening of a space for engagement between women volunteers from the community and block and district-level officials – space that did not exist until then. Following the submission of a report with the findings of the first nine months of reporting, local authorities agreed to establish regular grievance redressal forums so that women volunteers could meet with block health authorities and discuss the issues that emerged through reporting. The data also fed into the filing of written complaints through the government’s administrative system, which sought to address specific instances of rights violations, and, into the filing of right to information applications that sought to uncover bottlenecks in the disbursement of funding for health and nutrition services to lower-level facilities.

Conclusion

The larger accountability for economic, social, and cultural rights field offers lessons regarding how NGO or citizen-led engagement can foster accountability for the progressive realization of health care service delivery that is available, accessible, acceptable, and of adequate quality.²² We contribute to that by focusing on operational questions arising among practitioners trying to leverage ICT for use in right to health projects among particularly marginalized populations.

Despite obvious challenges in using ICT tools, poorer and less technically savvy members of the community have been able to register a significant volume of complaints based on their experience of using public health services. With the anonymity provided through these NGO mechanisms, women users were protected from the risk of retaliation by the providers and avoided intimidating bureaucratic encounters. The NGOs in turn were able to collect and analyze reports to identify recurring patterns and systemic flaws, which would have been more difficult using pen and paper. The real-time data gathered through ICT could be easily shared with government officials and the media for greater impact; where relevant, databases could also function as case management tools.

However, several factors deterred reporting: complaints did not usually lead to immediate improvement, some community members did not believe that their reports would lead to change, and some were reluctant to blame under-resourced frontline providers. The NGOs were hampered by technical challenges in these ICT-dependent projects and had not necessarily planned for the considerable financial and human support required to keep the projects running smoothly.

All three projects needed to conduct a number of offline activities in order to build trust with the health managers and providers, to foster rights and entitlement consciousness within the communities addressed by the project, and to convince community members of the importance of reporting. Uptake of the ICT tools was enhanced by these concerted efforts to engender government responsiveness while simultaneously building up an informed and empowered collective of users of public services.

These process findings have implications for the potential impact of ICT-enabled human rights accountability projects. The preliminary findings on impact were that while citizens overcame their fear and reluctance to report violations, many providers and managers did not respond adequately to address the systemic flaws that became apparent via the reporting. Health managers and policy actors remained constrained from providing a comprehensive system response or were unwilling to do so.

The standard bureaucratic approach of governments is to register individual complaints and seek to punish proven transgressions; this fails to transform the system or enhance answerability or enforceability for the users. The transformative potential of the projects more likely lies in the “micro-transgressions” of gender, caste, and other oppressive norms; the increased self and collective efficacy of those making reports; and the creation of opportunities for excluded people to interact meaningfully with government duty-bearers.²³ These changes may be a link in the causal chain that ends in increased responsiveness. If properly supported as a program component, ICT can facilitate linkages in the chain.

Acknowledgments

COPASAH provided an institutional umbrella for this research. Amy Manning and Anaise Williams of Columbia University provided research assistance. Lynn Freedman of Columbia University provided intellectual stewardship. Sandhya Y.K. of SAHAYOG provided important input into the SAHAYOG research. Katharina Wahedi contributed to the analysis of CEGSS research.

Marta Schaaf is the director of Programs and Operations at the Program on Global Health Justice and Governance in the Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University, New York, NY, USA. At the time this research was conducted, she was the Deputy Director of the Averting Maternal Death and Disability Program, also in the Heilbrunn Department of Population and Family Health.

Shruti Chhabra is a doctoral student in the Department of Sociomedical Sciences at the Mailman School of Public Health, Columbia University, New York, NY, USA.

Walter Flores is the director of the Centro de Estudios para la Equidad y Gobernanza en los Sistemas de Salud-CEGSS, Ciudad Guatemala, Guatemala.

Francesca Feruglio is co-founder of Nazdeek, New York, NY, USA.

Jashodhara Dasgupta, National Foundation for India, Core 4A (Upper Ground Floor), India Habitat Centre, New Delhi, India.

Ana Lorena Ruano, Center for International Health, Jussbyg II, Jekkteviksbakken 31, Bergen, Norway.

Please address correspondence to the authors c/o Marta Schaaf. Email: mls2014@cumc.columbia.edu.

Competing interests: None declared.

Copyright © 2018 Schaaf, Chhabra, Flores, Feruglio, Dasgupta, Ruano. This is an open access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.

References

Siddiqi, T. I. Masud, S. Nishtar, D. H. Peters, B. Sabri, K. M. Bile, and M. A. Jama, “Framework for assessing governance of the health system in developing countries: gateway to good governance,” Health Policy 90/1 (2009), pp. 13-25; T. Vian, “Review of corruption in the health sector: Theory, methods and interventions,” Health Policy and Planning 23/2 (2008), pp. 83-94; M. N. Marshall, P. G. Shekelle, S. Leatherman, and R. H. Brook, “The public release of performance data: What do we expect to gain? A review of the evidence,” JAMA283/14 (2000), pp. 1866-1874; World Bank, World development report 1993: Investing in health (New York: Oxford University Press, 1993); G. Bloom, “Equity in health in unequal societies: meeting health needs in contexts of social change,” Health Policy 57/3 (2001), pp. 205-224; World Health Organization, Everybody’s business–strengthening health systems to improve health outcomes: WHO’s framework for action (Geneva: WHO, 2007).
A. C. Pico, J. C. Kohler, J. Hoffmann, and L. Mungala, “No more broken promises: challenges and opportunities for key populations in demanding more transparency, accountability, and participation in the global response against the HIV and AIDS epidemic,” Health and Human Rights, 19/2 (2017), p.197; UN Committee on Economic, Social and Cultural Rights (CESCR), General Comment No. 14: The Right to the Highest Attainable Standard of Health (Art. 12), August 11, 2000, E/C.12/2000/4.
Pico (see note 2); D. Berlan and J. Shiffman, “Holding health providers in developing countries accountable to consumers: a synthesis of relevant scholarship,” Health Policy and Planning27/4 (2011), pp. 271-280; E. Lodenstein, M. Dieleman, B. Gerretsen, and J. E. Broerse, “A realist synthesis of the effect of social accountability interventions on health service providers’ and policymakers’ responsiveness,” Systematic Reviews 2/1 (2013); J. Gabbay and A. le May, “Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care,” BMJ 329/7473 (2004); C. B. Camargo and E. Jacobs, Social accountability and its conceptual challenges: An analytical framework (Freiburg: Max Planck Society for the Advancement of Science, 2013); T. I. M. Scott, R. Mannion, H. T. Davies, and M. N. Marshall, “Implementing culture change in health care: theory and practice,” International Journal for Quality in Health Care 15/2 (2003), pp. 111-118.
Fox, J. Aceron, and A. Guillán, Doing accountability differently. A proposal for the vertical integration of civil society monitoring and advocacy (Bergen: Chr. Michelsen Institute, 2016); J. Dasgupta, Y. K. Sandhya, S. Lobis, P. Verma, and M. Schaaf, “Using technology to claim rights to free maternal health care: Lessons about impact from the My Health, My Voice Pilot Project in India,” Health and Human Rights 17/2 (2015), pp. 135-147; S. Carlisle and S. Cropper, “Investing in lay researchers for community-based health action research: Implications for research, policy and practice,” Critical Public Health19/1 (2009), pp. 59-70; A. Appadurai, “The right to research,” Globalisation, Societies and Education 4/2 (2006), pp. 167-177; UN Committee on Economic, Social and Cultural Rights (see note 2).
Lodenstein et al. (see note 3); M. Schaaf, S. M. Topp, and M. Ngulube, “From favours to entitlements: community voice and action and health service quality in Zambia,” Health Policy and Planning 32/6 (2017), pp. 847-859; S. A. Papp, A. Gogoi, and C. Campbell, “Improving maternal health through social accountability: A case study from Orissa, India,” Global Public Health8/4 (2013), pp. 449-464; S. Gullo, C. Galavotti, and L. Altman, “A review of CARE’s Community Score Card experience and evidence,” Health Policy and Planning 31/10 (2016), pp. 1467-1478.
Hrynick and L. Waldman, ICT-facilitated accountability and engagement in health systems: a review of Making All Voices Count mHealth for accountability projects, Making All Voices Count Research Report (Brighton: Institute of Development Studies, 2017); S. Rebecca, K. Rono-Bett, and S. Kenei, Citizen-generated data and sustainable development: evidence from case studies in Kenya and Uganda, Making All Voices Count Research Report (Brighton: Institute of Development Studies, 2017).
Schedler, “Conceptualizing accountability,” in A. Schedler, L. Diamond, and M. F. Plattner (eds), The self-restraining state: Power and accountability in new democracies (Boulder, CO: Lynne Reinner, 1999), pp. 13-28; A. M. Goetz and J. Gaventa, Bringing citizen voice and client focus into service delivery, IDS Working Paper 138 (Brighton: Institute of Development Studies, 2001).
UN Committee on Economic, Social and Cultural Rights (see note 2).
Ibid; H. Potts, Accountability and the Right to the Highest Attainable Standard of Health (Essex: University of Essex, 2008).
Dasgupta et al. (see note 4); Hyrnick and Waldman (see note 6); T. Peixoto and J. Fox, “When does ICT-enabled citizen voice lead to government responsiveness?” IDS Bulletin 47/1 (2016), pp. 23-40; R. McGee with D. Edwards, C. Anderson, H. Hudson, and F. Feruglio, Appropriating technology for accountability: messages from Making All Voices Count, Making All Voices Count Research Report (Brighton: Institute of Development Studies, 2018).
W. Brinkerhoff, J. C. Hertz, and A. Wetterberg, “Introduction: Governance, social accountability, and sectoral service delivery,” in A. Wetterberg, D. W. Brinkerhoff, and J. C. Hertz (eds), Governance and service delivery: practical applications of social accountability across sectors (Research Triangle Park, NC: RTI International, 2016), pp. 1-8; J. Fox, “The uncertain relationship between transparency and accountability,” Development in Practice 17/4-5 (2007), pp. 663-671; A. M. Goetz and R. Jenkins, “Hybrid forms of accountability: citizen engagement in institutions of public-sector oversight in India,” Public Management Review3/3 (2001), pp. 363-383; M. S. Grindle and M. E. Hilderbrand, “Building sustainable capacity in the public sector: What can be done?” Public Administration and Development 15/5 (1995), pp. 441-463; A. Haines, S. Kuruvilla, and M. Borchert, “Bridging the implementation gap between knowledge and action for health,” Bulletin of the World Health Organization 82/10 (2004), pp. 724-731.
Hrynick and Waldman (see note 6).
Nazdeek, PAJHRA, and ICAAD, No Time to Lose: Fighting Maternal Mortality in Assam. Available at https://issuu.com/endmmnow/docs/endmmnow_advocacy_final_press; K. Wahedi, W. Flores, C. Beiersmann, K. Bozorgmehr, and A. Jahn, “Using information communication technology to identify deficits in rural health care: a mixed-methods evaluation from Guatemala,” Global Health Action 11/1 (2018); J. Dasgupta, “Ten years of negotiating rights around maternal health in Uttar Pradesh, India,” BMC International Health and Human Rights 11/3 (2011).
Wahedi et al. (see note 13); A. Das and J. Dasgupta, Claiming entitlements: The story of women leaders’ struggle for the right to health in Uttar Pradesh, India (New Delhi: Centre for Health and Social Justice, 2015).
Flyvbjerg, “Five misunderstandings about case-study research,” Qualitative Inquiry 12/2 (2006), pp. 219-245.
Hrynick and Waldman (see note 6); Peixoto and Fox (see note 10).
J. Hernández, “Los vigilantes de la salud,” El Periodico (June 11, 2017). Available at https://elperiodico.com.gt/domingo/2017/06/11/los-vigilantes-de-la-salud/.
Peixoto and Fox (see note 10); H. F. Spirer, “Violations of human rights—how many? The statistical problems of measuring such infractions are tough, but statistical science is equal to it,” American Journal of Economics and Sociology 49/2 (1990), pp. 199-210.
Srivastava, B. I. Avan, P. Rajbangshi, and S. Bhattacharyya, “Determinants of women’s satisfaction with maternal health care: a review of literature from developing countries,” BMC Pregnancy and Childbirth15/1 (2015), p. 97; S. Gabrysch and O. M. Campbell, “Still too far to walk: literature review of the determinants of delivery service use,” BMC Pregnancy and Childbirth 9/1 (2009), p. 34; Wahedi et al. (see note 13).
Dasgupta and V. A. Beard, “Community driven development, collective action and elite capture in Indonesia,” Development and Change38/2 (2007), pp. 229-249; J. F. Lund and M. Saito-Jensen, “Revisiting the issue of elite capture of participatory initiatives,” World Development 46 (2013), pp. 104-112.
Dasgupta et al. (see note 4).
Marshall, K. Ward, and N. Browne, “Reimagining rights-based accountability: Community use of economic and social rights,” The Irish Community Development Law Journal 3/1 (2014), pp. 65-81; S. Van Belle, V. Boydell, A. S. George, et al., “Broadening understanding of accountability ecosystems in sexual and reproductive health and rights: A systematic review,” PLoS ONE 13/7 (2018), p. e0200675.
Scott, A. S. George, S. A. Harvey, S. Mondal, G. Patel, and K. Sheikh, “Negotiating power relations, gender equality, and collective agency: Are village health committees transformative social spaces in northern India?” International Journal for Equity in Health16/1 (2017), p. 84.