SmartAid Index

Technical Guide

Mayada El-Zoghbi

Emmanuelle Javoy

Barbara Scola

© 2014 Consultative Group to Assist the Poor/The World Bank

All rights reserved.

The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The Consultative Group to Assist the Poor encourages dissemination of work and will normally grant permission promptly.

Consultative Group to Assist the Poor

1818 H Street, N.W.

Washington, DC 20433

Internet: www.cgap.org

Email: cgap@worldbank.org

Telephone: +1.202.473.9594

Contents

I. SmartAid Index: A Brief Overview

II. What Does SmartAid Offer Funders?

III. SmartAid Methodology

IV. Three Options to Use the SmartAid Index

V. Implementing SmartAid Step by Step

VI. SmartAid Indicators

VII. Resources

Annex I: Terms of Use of the SmartAid Index

Annex II: Compact for Better Aid for Access to Finance

I. SmartAid Index: A Brief Overview

The SmartAid Index (SmartAid) measures whether funders are set up to support financial inclusion effectively. It assesses a funder’s internal systems and practices related to its financial inclusion projects—meaning all projects that contribute to a state in which all households and businesses have access and the ability to use a broad range of appropriate financial services that are provided responsibly and sustainably in a well-regulated environment.

Measuring the effectiveness of funding for financial inclusion is not easy, given the long chain from project[1] design, to implementation on the ground, to impact on the lives of the poor. SmartAid focuses on the first part of this chain, funders’ internal management systems, with the simple premise that funders with strong systems are more likely to deliver good projects on the ground. Many factors that affect project performance are beyond funders’ influence, thus SmartAid focuses on those factors where funders have more direct control—their internal systems. SmartAid does not evaluate the performance of funders’ financial inclusion projects on the ground; other tools (e.g., portfolio reviews) exist for that purpose.

In 2006, 29 major development institutions[2] asked CGAP to create a tool that would help them improve the effectiveness of their support to microfinance. In response, CGAP developed the SmartAid Index. Twenty development institutions participated in SmartAid between 2007 and 2013, representing the major funders supporting financial inclusion (see Table 1).

In the past, the SmartAid Index was offered every two years. It was fully administered by CGAP and relied on an external review board consisting of four independent financial inclusion specialists. Going forward, the SmartAid Index is offered as a tool that funders can use with support from certified consultants who have all participated in the previous external review board and have been certified by CGAP. This new approach allows funders to implement SmartAid anytime, whenever it is strategic and feasible for them. It also makes SmartAid accessible to a larger array of funders.

Over the years, the methodology was refined to focus on the most relevant aspects of funders’ internal systems, but also on those that can objectively be compared across different institutions. Although SmartAid evolved to reflect changes in the industry, for example, the shift from microfinance to a broader vision of financial inclusion, the essence of SmartAid hasn’t changed.

While changing internal systems takes time, the results from past SmartAid rounds are translating into concrete improvements.[3] For more information on SmartAid and funders’ SmartAid reports, visit www.cgap.org/about/programs/smart-aid.

This Technical Guide is aimed at funders interested in analyzing their effectiveness in financial inclusion. It describes the methodology of the SmartAid Index and explains how to use SmartAid.

II. What Does SmartAid Offer funders?

The SmartAid Index is a service for all types of funders that want to improve their effectiveness in supporting financial inclusion. Past SmartAid participants include development finance institutions (DFIs), multilateral development agencies, regional development banks, and bilateral agencies using different instruments, such as grants, debt and equity investments, or loans to governments.

The main benefits of SmartAid include the following:

•   Helps identify internal best practices and areas for improvement. Institutions can use SmartAid to understand where they are strong and where they need improvement to support financial inclusion projects effectively. It can be used to draw management attention to issues that may have been overlooked and helps identify priorities for change.

•   Spurs internal dialogue on improving effectiveness. Funders can use SmartAid to facilitate an internal dialogue about effectiveness in supporting financial inclusion. Implementing SmartAid requires collaboration across the organization and helps spark discussions among different departments that work on financial inclusion or between staff and management.

•   Allows funders to compare their performance with peers. SmartAid helps funders understand where they stand relative to other funders. This provides opportunities for peer learning and collaboration. It also helps funders track their progress over time and compare progress with that of their peers.

•   Provides evidence of commitment to effectiveness. By participating in SmartAid, funders show their commitment to effectiveness to both internal and external stakeholders. Funders who validate and publish their SmartAid results demonstrate their commitment to transparency and accountability, which sends a strong signal to partners, stakeholders, and peers.

III. SmartAid Methodology

The SmartAid methodology was developed by CGAP in collaboration with a broad range of funders and financial inclusion experts and advisers on aid effectiveness (including the Center for Global Development and OECD/DAC). The methodology builds on the findings of 17 Microfinance Donor Peer Reviews facilitated by CGAP from 2002 to 2004. From this work, five elements emerged as critical for the effectiveness of funders in financial inclusion. These five elements where translated into a set of indicators that constitute SmartAid (see Table 2).

Funders can reach a maximum of 100 points in the SmartAid Index. Indicators account for either 10 or 15 points depending on their relevance for a funder’s overall effectiveness in financial inclusion. This results in different weights for the five elements of effectiveness. Accountability for results accounts for 40 percent of the total score, followed by staff capacity (25 percent), strategic clarity (15 percent), and knowledge management and appropriate instruments (10 percent each).

Depending on their performance, funders reach a score ranging from 0 to 5 for each indicator. A funder would receive a score of 0 on an indicator if it had no systems in place. A score of 5 means the funder provides an example of good practice on this indicator. The scores are weighted (resulting in either 10 or 15 points per indicator), and added up to calculate the total score.

One objective of SmartAid is to show how funders progress over time and how their performance compares to others, which requires keeping a constant set of indicators. The set of indicators was streamlined significantly after the SmartAid pilot round in 2007. The scores are comparable among agencies having participated in the SmartAid Index since 2009.

IV. Three Options to Use the SmartAid Index

There are three basic options to use SmartAid, which can be further customized to match funders’ needs and expectations.

•   SmartAid Index self-assessment. The funder uses SmartAid without external assistance based on a toolkit provided by CGAP. This option is most useful for agencies that are exploring the process but not yet committed to public disclosure. A self-assessment spurs internal discussion and helps the agency identify internal strengths and weaknesses; however, there is no external assessment included in the process. The score received with this option is comparable with other SmartAid scores, but is not validated by CGAP and will not be integrated into industry benchmarks.

•   SmartAid Index accompanied self-assessment. The funder receives support from a certified SmartAid consultant to conduct the self-assessment. This option is useful for agencies that have limited staff capacity and may need support to facilitate the internal process required to implement SmartAid. As with the self-assessment, the focus is on internal discussion and generating knowledge around what it takes to improve internal practices. The support of the SmartAid consultant will also ensure that the funder gets information on the practices implemented by other agencies on the indicators that might need improvement. The score received with this option is comparable with other SmartAid scores, but is not validated by CGAP and will not be integrated into industry benchmarks.

•   SmartAid Index validated assessment. The funder receives support from a certified SmartAid consultant to conduct the SmartAid assessment, and the results are validated by a second certified consultant and CGAP. This option is useful for agencies that are seeking support throughout the process, that want to benchmark their performance against their peers, and that would like ultimately to publish their score. A detailed report provides recommendations for improvement. The score and the report are validated by CGAP and are published on www.cgap.org.

Funders can conduct a SmartAid assessment anytime of the year, depending on their demand. The SmartAid process will take up to three months. When considering SmartAid, funders are recommended to check potential complementarities (or overlaps) with ongoing internal evaluations or upcoming strategy development processes. The results from SmartAid are a very useful input for such processes, even beyond the field of financial inclusion.

V. Implementing SmartAid Step by Step

The SmartAid Index is based on an assessment of a funder’s internal management systems according to a set of nine indicators. The central piece of the assessment is a questionnaire that captures a funder’s performance against those indicators. Responses provided in the questionnaire have to be supported with evidence and, depending on the implementation option chosen, are discussed in an internal workshop or validated by external reviewers.

Implementing SmartAid includes four phases: (1) preparation, (2) collecting evidence, (3) completing the questionnaire, and (4) analyzing results. Depending on the implementation option chosen by the funder, the steps within those phases vary (see Figure 1). The following section provides guidance on each step in the process.

1. Preparation

Funders interested in participating in the SmartAid Index are asked to register on the CGAP website at http://www.cgap.org/about/programs/smart-aid.

The registration does not lead to any obligation from the funder’s side. CGAP will contact interested funders and advise them on the implementation option adapted for them. Once participation is confirmed, and ideally endorsed by a funder’s senior management, the funder receives the SmartAid Toolkit, which includes all the necessary materials to implement SmartAid. (See Box 1.)

Box 1. SmartAid Toolkit

•   SmartAid Technical Guide

•   Questionnaire on SmartAid indicators (Excel)

•   “At a Glance,” a form used to collect general information about the institution, instruments, and project/investment cycle (Word)

•   Access to online staff survey (surveymonkey)

•   Standard terms of reference and list of certified consultants, including short biographies

Implementation of SmartAid is ideally coordinated by the financial inclusion focal point within a funding agency. As the usefulness of SmartAid depends significantly on the involvement of staff working on financial inclusion throughout an organization, the focal point gathers a group of three to six key staff to form the SmartAid working group. This group will be responsible for steering the process and for completing the questionnaire. The focal point should consider inviting colleagues with the following functions to be part of the working group: technical support/advisory, operations and/or investments (e.g., financial sector department, agriculture department, or human development department), as well as risk management, contracting, procurement, monitoring and evaluation.

2. Evidence

SmartAid focuses on funders’ internal systems. The nature of these systems is codified, or at a minimum reflected, in different types of documents, such as strategy/policy documents, operational guidelines, job descriptions, and contracts or agreements. For SmartAid to result in a significant assessment that is not based just on individual judgment, a funder’s performance needs to be supported by documented evidence. The collection of documents is also a necessary step to provide informed responses to the questionnaire. Funders might also find this step useful to constitute a comprehensive collection of documents related to their internal practice in financial inclusion, which can easily be shared with new staff, for example.

The types of documents that reflect internal systems differ from one organization to another. It is important for funders to understand what each indicator is trying to measure based on the description in this guide (see Chapter VI, SmartAid Indicators) and to provide the most appropriate documents as evidence for internal practice.

Funders opting for the accompanied or the validated assessment will be assisted in this step by a certified consultant. The consultant will work with the funder’s staff by phone to explain the SmartAid process and indicators and provide guidance to identify the documents required for evidence about internal systems.

Funders opting for the accompanied or the validated assessment are required to upload the documents to a secured online platform where they are available to the consultants and CGAP.

Not all internal systems are captured in documents, and paper documents do not always reflect actual practice. An online staff survey is therefore used as additional evidence on how a funder works in financial inclusion. The survey includes some of the questions also included in the questionnaire, focusing on those where staff perceptions are likely to vary. Access to the online survey is provided by CGAP as part of the SmartAid Toolkit.[4] The raw results of the online survey are provided to the funder for analysis.

The SmartAid working group invites a group of staff to participate in the survey. This group should represent the variety of persons involved in financial inclusion projects throughout the organization, including managers and staff that are directly or indirectly involved in design, management and contracting, and monitoring and evaluation of financial inclusion projects. Persons from different thematic or regional departments should be included, as well as relevant staff from headquarters and country offices. Staff in charge of risk management and procurement should also be included. The broader the sample, the more representative and useful the result will be.

Funders opting for the accompanied or the validated assessment can ask for assistance from a certified consultant to constitute the participants list for the survey, to follow-up with participants, and to analyze the results.

3. Questionnaire

The questionnaire is provided by CGAP in the form of a protected Microsoft Excel sheet. The funder’s SmartAid working group completes the questionnaire based on the information provided in the documents and by the staff survey. This ensures that responses reflect internal practice and are not based only on individual judgment. All questions in the questionnaire must be answered, and each answer should be explained in a short justification.

The questionnaire includes between 5 and 10 questions per indicator and has a total of 65 questions. For each question, there are five answer options to choose from. It is recommended that the answer be selected by consensus among the funder’s SmartAid working group. An average of 5 minutes of discussion per question should be sufficient to reach consensus (i.e., a total of 5 to 6 hours to fill in the questionnaire). If consensus is difficult to reach on some questions, it is likely that further information needs to be gathered. It is recommended to organize two half-day sessions to complete the questionnaire.

A workshop is organized with the members of the funder’s SmartAid working group to discuss and finalize responses to the SmartAid questionnaire based on inputs from the staff survey and collected documents.

In the accompanied and validated assessment, the consultant can be asked to organize and facilitate the workshop. The consultant will assist funder staff to reach consensus on the responses to the questionnaire. The workshop can also be used to discuss the results of the staff survey in more detail.

Funders opting for the validated assessment submit the completed questionnaire to the certified consultant. The consultant reviews the questionnaire in detail and checks the evidence for each answer based on the submitted documents, the staff survey, and the discussions during the workshop. The consultant also takes into account the responses provided by the SmartAid working group and the justifications in the questionnaire. The questionnaire is then validated by a second certified consultant who has no ties to the funder and was not involved in facilitating the process. The validator also has access to the submitted documents and the results of the staff survey. Both consultants will coordinate to agree on final responses to the questionnaire. Where there is disagreement with the responses submitted by the funder, the consultants may need to follow up with the funder through phone interviews with the financial inclusion focal point or the funder’s SmartAid working group to clarify additional questions. The final version of the questionnaire is reviewed by the SmartAid working group and then submitted to CGAP.

4. Results

To receive their SmartAid scores, funders must submit the completed questionnaire to CGAP. By submitting their questionnaire, funders agree that CGAP can quote them publicly as “SmartAid participants.” However, only validated scores are used to calculate industry benchmarks.

No later than two weeks after submitting the completed questionnaire, funders receive their results in the form of graphs that show the overall score and the score on each indicator. The results highlight the funder’s scores compared to the industry benchmarks showing the median, minimum, and maximum scores of all funders having participated in SmartAid to date (see Figure 2). The scores help funders identify strengths and weaknesses in their internal management systems and show where they stand relative to others.

In the validated assessment, funders receive a report that consolidates the learnings from SmartAid and can be shared with internal and external stakeholders. The report summarizes internal strengths and weaknesses, highlights a good practice example, and includes a section with recommendations for improving the effectiveness in supporting financial inclusion.

The report is drafted by the certified consultant who facilitated the process. The consultant takes into account feedback from the other certified consultant and CGAP. The funder’s SmartAid working group reviews the report to make sure there are no factual inaccuracies. The final report is published on CGAP’s website. Funders that do not want to make their reports public can ask for an exemption in written form (email to CGAP).

The funder’s SmartAid working group is encouraged to organize a debriefing workshop to discuss the results of the assessment with staff and management. This workshop is an opportunity to share the results with staff who participated in the process, to draw management attention to internal good practices but also to those areas requiring change, and to discuss an action plan to improve effectiveness in supporting financial inclusion.

In the accompanied and validated assessment, the consultant can be asked to prepare and facilitate the debriefing workshop and to draft a summary of the main findings.

VI. SmartAid Indicators

This section provides a brief explanation of the rationale and relevance of the nine indicators. For each indicator, the explanation is followed by a detailed listing of what SmartAid looks for in funders’ policies and practices, as well as specific examples of documents a funder might submit to demonstrate the state of practice within its agency.

1.  Funder has a policy and strategy that addresses financial inclusion, is in line with good practice, and is based on its capabilities and constraints.

A coherent vision improves quality at every level of an institution. A policy document articulates principles, guidelines, or standards that set a definite and binding common approach for reaching a funder’s vision. In addition to laying out funders’ visions and goals for financial inclusion, policy documents can also make the link between financial inclusion and the institution’s overall development mission.

Strong policies and a clear vision are mutually reinforcing. Funders with strong policies that adhere to international standards embrace a financial systems development approach rather than emphasizing credit as an input or resource transfer. A policy does not dictate one way to support financial inclusion; rather it supports diverse approaches and priorities within a framework of basic good practice principles.

A funder’s strategy for financial inclusion brings its policy into focus. Strategies for financial inclusion may be agencywide or at the regional or country level. They might be integrated in a financial sector strategy or they may stand alone. Reflecting on internal capabilities and constraints can help funders formulate their strategies for supporting financial inclusion projects. A consideration of the broader funding landscape and where a funder can add value in relation to other donors and investors is also needed given the growing numbers of funders and the increasing pace of change in the financial inclusion space. Not every funder needs to—or can—tackle all gaps and challenges to building inclusive financial systems. Also, funders need to make sure their interventions are in line with local needs and the stage of maturity of any given market. Rather than proposing a one-size-fits-all approach, a funder’s strategy or operational guidelines should ensure interventions are based on a thorough market assessment and an analysis of other funders’ activities in the same market.

Strategies that aim to develop inclusive financial systems that are sustainable in the long run require that a funder’s activities respect additionality—a key principle in private sector development. Additionality is about whether a funder is producing or catalyzing a positive change that otherwise would not happen or would take much longer to happen. For example, is the funder filling a gap that would otherwise not be covered by local or other commercial funding sources? Strategies that prevent funders from distorting local markets and encourage them to leverage socially responsible investments or commercial capital are likely to have a stronger development impact.

Finally, strong support from senior management for policies and strategies and a clear signal that they are meant to be followed increase the likelihood that these policies and strategies will be internalized by staff, applied in their everyday work, and thus translated into results on the ground.

SmartAid looks for the following:

•   Policies that are in line with good practices.

•   Policies that are endorsed by management. Official endorsement by the relevant management level sends a strong signal to staff that a policy is to be followed.

•   Policies that are binding. SmartAid considers that only binding policies have a substantial influence on the quality of microfinance projects and investments. Whether a policy is binding depends not only on whether it is officially endorsed, but also on whether it provides mandatory guidelines for its implementation beyond theoretical suggestions, and whether there are mechanisms to check and enforce compliance.

•   Policies that take into account current issues and evolutions in financial inclusion. Given how quickly the landscape is evolving, policies have to be reviewed and adapted periodically to remain relevant. Policies that take current issues into account and reflect the impact of new actors (i.e., entry of socially responsible investors or mobile network operators) and developments (i.e., technological advances, new industry standards such as client protection principles) serve as a relevant guide to staff and management.

•   Policies that are broadly disseminated and accessible to staff. Policies are more effective if they are communicated broadly to staff and are easily accessible on the funder’s website or intranet. Internal trainings or orientation sessions for new staff are good opportunities to communicate policies.

•   Policies that are user friendly. Policies that are centered on the needs of operational staff and include clear operational guidance that can be used throughout the project/investment cycle (e.g., what to do, what not to do; templates; checklists; etc.) are more likely to be consulted by staff during project/investment design and implementation.

•   Strategies that are based on realistic analyses of internal strengths and weaknesses, as well as external opportunities and threats (SWOT).

•   Strategies that demonstrate an awareness of the broader funding landscape and emphasize the importance of additionality and responsiveness to market conditions. Strategies that drive funders to leverage commercial funding and prevent crowding out of domestic funding sources are more effective in building inclusive financial systems.

Documents to be provided as evidence:

Strategy/policy document for financial inclusion—can be part of a broader financial sector development/private sector development or regional strategy/policy.

Operational guidelines for staff providing guidance to implement the strategy.

2.  Funder has quality assurance systems in place to support financial inclusion projects and investments.

Experience suggests that funders perform best when project/investment teams include financial inclusion specialists or when projects are at least reviewed by these technical specialists.

Key decision points for quality assurance in the project/investment cycle include project design (prior to approval), approval, and monitoring and evaluation. Quality reviews that come early enough so that they can influence the design of the project/investment are the most effective, and, ideally, a specialist is included in the design team from the earliest phases. Review prior to approval is especially important for financial sector components of nonfinancial projects as they are often designed by staff without financial inclusion expertise. Ensuring strong technical inputs throughout the project/investment cycle is one way to counteract approval and disbursement pressure that is prevalent in many funding agencies.

The way quality assurance is implemented may differ depending on the organizational type, structure, and size of the funding agency. Ideally, projects are designed by staff with extensive expertise. In addition, projects are reviewed by a different person (or committee) that is not involved in the actual project management, bringing both financial inclusion expertise and an independent view on the quality of the project design.

In organizations that are highly decentralized and at the same time do not have a centralized quality assurance department, quality assurance typically relies on the capacity of technical specialist staff who are involved in all key steps of the project/investment cycle, regardless of where they are based (e.g., at headquarters or in country offices). In this case, quality assurance involves making sure those specialists have the skills and tools they need to design and manage high-quality projects. However, even in decentralized organizations, projects should be reviewed by a person other than the project manager.

In agencies where generalists design projects, quality can be enhanced if financial inclusion specialists are mandated to provide inputs and technical review throughout the project/investment cycle. In such cases, there should be strong incentives for project managers to follow the specialists’ recommendations (e.g., through involvement in managing the review process or through a requirement to justify deviations from technical recommendations).

Quality assurance processes should involve reviewing the following:

•   Coherence of project with funder’s financial inclusion strategy and national/regional priorities

•   Adherence to international best practices

•   Whether project design is based on a thorough assessment of local market needs

•   The value addition of the funder, and how it has been assessed (e.g., review of local funding landscape and funder’s comparative advantage)

•   Whether proposed funding instruments are in line with project objectives and structured and priced according to market conditions

SmartAid looks for the following:

•   Quality assurance processes that require the participation of staff with sufficient technical skills in project design and/or review, as well as in monitoring of financial inclusion projects and investments.

•   Quality assurance provided by staff who are formally designated to do so. Designations for the function of providing early technical inputs into project design or technical review—rather than portfolio management—can be formalized in terms of reference (TOR) of staff and/or in the description of a specific unit or group.

•   Mechanisms for quality assurance that match a funder’s organizational structure and are commensurate with the financial inclusion portfolio and the extent of expertise throughout the institution.

•   Strong incentives to follow technical advice. Mandating that technical advice is taken into account is likely to result in the application of good practices at the project/investment level. The funder’s processes should ensure that project managers have to justify cases where technical advice is not followed.

Documents to be provided as evidence:

Project/investment cycle according to template provided by CGAP.

Organizational chart that shows all different units working on financial inclusion and explains their role.

Job descriptions, TOR, and/or selection criteria of staff formally designated as responsible for quality assurance.

3.  Funder has the staff capacity required to deliver on its financial inclusion strategy.

Staff capacity is undoubtedly a critical success factor in the design, implementation, and monitoring of any project or investment. To successfully implement its strategy a funder needs a sufficient number of staff with the right skill mix to design, manage, and monitor projects and investments.

The number of qualified staff members needed to manage financial inclusion projects or investments depends on the institutional structure of the funder; the nature of its quality assurance mechanisms (indicator 2); and the size, composition, and dispersion of its portfolio. For example, a decentralized agency that counts on its field staff for ensuring the quality of its projects will need many more field-level staff with the appropriate skills and training than another agency might.

Just as important as the number of qualified staff is whether their experience and skills match the funder’s strategy and portfolio. Staff responsible for funding retail microfinance institutions need different skills than staff advising central banks on regulation and supervision of the financial sector. In the best case scenario, the number and mix of staff working on financial inclusion and the skills and experience they offer are consistent with the type of interventions supported, the size of the overall portfolio, and the extent of financial inclusion expertise throughout the agency.

Building and maintaining staff capacity is an ongoing process and can be approached in many ways. Some funders have recruitment policies that prioritize hiring specialists with significant technical expertise. Others hire staff with less specific experience but have mandatory training for all new staff working on financial inclusion. Even experienced specialists need regular training to keep up with the latest evolutions in financial inclusion (e.g., mobile banking, microinsurance, savings services, etc.).

Beyond qualified project or investment staff, some level of financial inclusion knowledge or “basic literacy” is required within other functional units in an institution—this is often the case for risk management or evaluation departments, for example. Staff in these departments also have a say during the life cycle of projects or investments, and their opinions and decisions can influence current or future projects.

Funders that do not have sufficient capacity in-house can use external consultants to fulfill the role of technical specialists. Using a regular set of consultants or a database of preselected consultants can help compensate for insufficient in-house capacity. Nevertheless, having in-house specialist staff has certain advantages in terms of continuity and knowledge management over time. Even for funders that rely heavily on consultants, a minimal level of technical knowledge is required in-house. Basic financial inclusion knowledge is needed to select qualified consultants, ask the right questions, and interpret performance reports.

SmartAid looks for the following:

•   Funders with a sufficient number of qualified staff. The number of specialists required depends on the institutional structure of the funder; the nature of its quality assurance mechanisms (indicator 2); and the size, composition, and dispersion of its financial inclusion portfolio.

•   Strong match between the skill sets of staff and a funder’s portfolio and strategic directions in financial inclusion.

•   Funders that maintain staff with technical expertise in-house, whether at headquarters or in regional offices, to facilitate access to technical support.

•   Funders that do not have sufficient capacity in-house require the procurement flexibility, budget, and network to contract external consultants as technical specialists.

•   The availability of ongoing training and skills-building opportunities for staff. The appropriate frequency and subjects of such opportunities depend on the funder’s technical skills base and the type of projects it supports. Increasingly, more specific specializations within financial inclusion are called for (e.g., savings, rural finance, policy work, etc.). Prioritizing training topics that match the funder’s strategy and the content of its portfolio is most effective. Opportunities can be provided internally or externally (e.g., through staff exchanges).

•   Funders that build and maintain basic financial inclusion knowledge throughout relevant departments (legal, procurement, evaluation, risk management, etc.) either through recruitment policies or by training staff.

Documents to be provided as evidence:

List and CVs of technical specialists indicating the share of their time spent on financial inclusion.

4.  Funder has a system in place that identifies all financial inclusion projects and components.

An important first step in ensuring accountability for results is for funders to know about all their financial inclusion projects or investments, whether standalone or components. Simply identifying the full extent of the portfolio can be a challenge. For example, in many institutions, projects originate in more than one department or sector, and different units are involved over the course of the project cycle. A standardized project/investment identification system is needed to ensure the funder has a good overview over the entire portfolio and can thus provide technical support when needed.

While most funders can identify standalone projects or investments, many do not capture financial-inclusion-related components within larger projects. As a result, these components often do not receive sufficient technical input and oversight. Project identification systems or databases that allow staff to assign multiple codes or markers to a project make it easier to capture such components.

Automated project identification systems that can generate updated lists of all financial inclusion projects and components on demand and cover the entire agency’s portfolio are most useful for effective portfolio management. Clear coding guidelines and compliance checks can be used to make sure the information in the system is accurate and project codes are assigned correctly.

SmartAid looks for the following:

•   Systems that can easily produce updated lists of the financial inclusion portfolio. The highest standard is for institutions to have lists that can be automatically generated at any point in time to provide timely information. Manual lists can also work, however, especially within smaller agencies with concentrated portfolios.

•   Participation in CGAP Funder Survey and comprehensiveness of reporting (e.g., reporting of disbursed amounts, allocation by levels, etc.).

•   Systems that provide lists that are complete, covering the whole institution, not just the financial sector department. Both standalone projects and components of financial and nonfinancial sector projects should be included.

•   Funders that periodically check that all relevant projects and components are being coded correctly so as to ensure that the list of projects or investments is accurate.

•   Funders that have clear guidelines for how to code financial inclusion projects.

Documents to be provided as evidence:

List of financial inclusion projects.

List of codes (and subcodes) that are used to identify financial inclusion projects and components and coding guidelines or instructions.

5.  Funder monitors and analyzes performance indicators for financial inclusion projects and investments.

Another important step in accountability for results is monitoring, analyzing, and reporting on performance. Beyond simply knowing what they are funding, funders gain by knowing the performance of their portfolio.

Projects that are required to report regularly on their performance are more likely to perform better. The more transparent the results, the more likely funders are to learn from successes and failures and to take corrective actions when needed. Funders can use performance information, measured against objectives, to decide whether to continue, replicate, or terminate a project.

Funders should not only track the performance of retail financial institutions they are funding directly, but they should also monitor the portfolio of intermediaries, such as microfinance investment vehicles (MIVs) or apexes. For interventions at the retail level, funders should use internationally agreed-on indicators to track the performance of retail institutions (Rosenberg 2009). For interventions at the market infrastructure and policy level, standards are only emerging. The SEEP Network has developed financial performance indicators for microfinance associations, “Measuring Financial Performance: Practitioners Guide for Microfinance Associations,” and IFC for measuring the effectiveness of credit bureaus, “Credit Bureau Knowledge Guide.” Where no standards exist, funders should develop their own indicators to measure performance.

Aggregating portfolio information at the regional level or for the entire financial inclusion portfolio allows a funder to analyze performance patterns or identify problems. Reporting to a managerial level also strengthens accountability for the results of projects or investments.

SmartAid looks for the following:

•   Requirement of mandatory performance monitoring.

•   Requirement of reporting, with reports provided to staff responsible for monitoring the project and to a managerial level.

•   Funders that collect information such as the minimum indicators that donors and investors should track (Rosenberg 2009). Industry standard definitions and calculations allow performance to be compared within the agency as well as with industry benchmarks.

•   Reporting at regular intervals, at minimum annually, with semi-annually or quarterly considered a more appropriate frequency.

•   Monitoring and reporting systems that include both funding provided directly to retail financial service providers and funding channeled through intermediaries or wholesale facilities, such as apexes or MIVs.

•   Monitoring and reporting systems that include both standalone projects and components.

•   Performance monitoring systems that take into account social performance/responsible finance indicators or qualitative proxies to get beyond the financial bottom line.

•   Funders that use existing standards and monitor performance of all types of projects/investments. Such monitoring can include qualitative and quantitative information.

Documents to be provided as evidence:

List of standard performance indicators that are tracked for financial inclusion projects and investments.

Read-only access to a database with performance information on financial inclusion projects and investments or printouts of reports.

List of performance indicators for projects at the market infrastructure and policy levels.

6.  Funder incorporates performance-based elements in standard agreements with partners.

Loan and grant agreements formalize the relationship between a funder and its partners and state the obligations of both parties. Typical loan and grant agreements with retail financial institutions define the amount and modalities of funding, the permitted use of funds, the type and frequency of reporting required, as well as standard covenants about suspension or termination of funding.

In addition, standard loan and grant agreements can be used to define the expected results from an engagement and to create positive or negative incentives to achieve those results. Agreements that incorporate such performance-based elements are more likely to lead to the results the funder and the partner institution want to achieve.

An agreement is considered performance-based when (i) it is as clear and specific as possible about the expected results and how they will be measured, and (ii) it strengthens incentives for good performance by defining benefits or sanctions that are tied to the achievement of the expected results (El-Zoghbi, Glisovic-Mezieres, and Latortue 2010). Performance-based elements in a funding agreement include the following:

•   Clear set of indicators and targets for performance on these indicators

•   Minimum performance or “threshold” performance that is allowable on some of the indicators

•   Funding in tranches to allow for an ongoing incentive mechanism

•   Clear sanctions for noncompliance once performance falls short of minimum levels

•   Bonuses or other rewards for maintaining and exceeding targets set

Funders that use performance-based agreements need the mechanisms to monitor and track performance as set forth in the agreements. Regular reporting is required against the targets set in an agreement so that problems can be identified and actions taken, if necessary. Of course, the effectiveness of performance-based agreements depends on a funder’s political will and ability to take action, including withholding payments or even terminating the agreement with nonperforming projects or partners. In cases of noncompliance for loan agreements, funders need to consider carefully that calling a loan may have broader consequences on the market and should use this option with care.

Agreements with all kinds of partners (apexes, microfinance institutions, technical implementers, and even governments) can include performance-based elements. The nature of the agreement, the performance targets, indicators, monitoring plan, and sanctions for noncompliance will differ based on the partner, the funding instrument, and the type of project or investment.

SmartAid looks for the following:

•   Systematic use of performance-based elements in agreements with financial institutions or wholesale finance institutions as a matter of policy or standard practice, for both standalone projects and components within larger projects.

•   Agreements that incorporate key performance-based elements, such as minimum performance targets, thresholds, and clear sanctions for noncompliance. Mechanisms that incentivize performance, such as payment in tranches and bonuses for meeting or exceeding performance targets, are also important features of performance-based agreements.

•   Clear designation of staff responsible for monitoring performance-based agreements.

•   Reporting that compares actual performance to the targets set in agreements.

•   The use of performance-based elements in agreements with all partners, such as technical service providers or government agencies.

•   Evidence of actions taken when partners do not meet minimum thresholds defined in the performance-based agreement. Actions taken are responsible and take into account their impact on the broader market.

Documents to be provided as evidence:

Standard template of performance-based loan and grant agreements.

Guidance to staff on the use of performance-based agreements.

7.  Funder regularly reviews the performance of its financial inclusion portfolio.

Evaluations of individual financial inclusion projects are key to accountability, generate lessons learned, and influence project implementation when they are done mid-term. Such evaluations, however, have limited influence on future operations or strategic decisions.

Reviews that go beyond disconnected evaluations of individual projects and cover a funder’s full portfolio, or an important cross-section thereof, can yield more meaningful learning. Understanding what worked well, what did not, and why can inform future project designs and strategic reorientations and thus improve performance.

Portfolio reviews have emerged as a methodology to analyze a funder’s performance throughout its entire financial inclusion portfolio. A portfolio review analyses the composition of the portfolio and the performance of individual projects across the portfolio to assess whether a funder delivers on its financial inclusion strategy.

Portfolio reviews can cover the full portfolio or examine a specific subset of projects (e.g., projects in a specific region, projects with a similar thematic approach, or projects funded through the same instrument—e.g., greenfields). See Table 3.

Several methodologies can be used to review the performance of a funder’s financial inclusion portfolio; the key is to carefully analyze actual performance information on individual projects with a standardized method of measuring the performance (e.g., scoring of individual project performance). Depending on the quality of a funder’s internal monitoring systems, portfolio reviews can rely to a certain extent on secondary data, such as monitoring reports or consolidated standard performance indicators (Rosenberg 2009). Most commonly, portfolio reviews include a comprehensive review of project documents combined with field visits.

Reviews can cover various time spans; those that take a long-term perspective can also cover learning from terminated projects. These assessments can be carried out as internal or external evaluations.

SmartAid looks for the following:

•   Assessments/reviews that cover a significant section of a funder’s portfolio and, at minimum, check performance, achievement of targets, and compliance with strategy and good practice principles. This includes both financial and social performance as benchmarked against the institution’s objectives.

•   Comprehensive reviews that are done regularly; somewhere between every three to five years as appropriate.

•   Reviews that assess the funder’s additionality when supporting financial inclusion projects and investments.

•   Reviews that touch on the quality of the funder’s work. While many factors influence results, funders should reflect on whether their own inputs (advice, design, etc.) made an effective and appropriate contribution.

•   Reviews that are conducted by technical specialists, whether internal or external.

•   Reviews that, once completed, are shared within the agency and trigger discussions and a management response around the results.

Documents to be provided as evidence:

Full reports of portfolio reviews conducted in the past five years or TOR of an upcoming portfolio review if one is in progress.

8.  Funder has systems and resources for active knowledge management for financial inclusion.

Knowledge management includes identifying, creating, disseminating, and using knowledge. It entails practices used to capture and learn from one’s own—and others’—experiences. Actively managing knowledge is above all an organizational value and requires strong support from senior management to become ingrained in the culture of a funding agency. Ensuring strong communication between staff based in headquarters and staff in country/regional offices facilitates the exchange of institutional knowledge and diverse experiences. Effective knowledge management requires appropriate tools, mechanisms, and incentives.

Knowledge management systems have to be adapted to the organizational structure, communication culture, and size of an institution. Tools and mechanisms typically used for active knowledge management include vehicles to share information (web-based platforms, regular meetings), incentives for staff to act as knowledge managers (include knowledge management in job descriptions, budget for knowledge management activities), policy of systematically capturing and sharing knowledge (policy to make project documents available to all staff), and the distilling of knowledge into products that are shared with staff (case studies, learning events).

SmartAid looks for the following:

•   Job descriptions that include knowledge management.

•   Policy or institutional culture that makes key project documents available on shared platforms for easy internal access (e.g., consultant reports, appraisal reports, evaluation reports, back-to-office memos, etc.).

•   Opportunities for face-to-face interactions among staff of different levels, locations (headquarters and field), etc.

•   Budgets to support knowledge management. Budgets can be used both for participating in internal events and taking advantage of others’ learning events and conferences, and supporting websites, newsletters, resource centers, communities of practice, collaboration platforms, etc.

Documents to be provided as evidence:

Agencywide policy on knowledge management (e.g., position paper on knowledge management, notes from management, etc.)

List or link to learning events and knowledge products.

9.  Funder has appropriate instrument(s) to support the development of local financial markets.

Funders have an important role to play in developing pro-poor financial market systems. They can act as market facilitators and help crowd-in mainstream investors and local capital. As development agencies, funders play a temporary role in a market and have to make sure they provide development additionality (i.e., they contribute to the development of a functioning market system where their interventions are not needed in the long term). When intervening in a market, funders have to reflect on whether their support contributes to market development, beyond benefitting the institution receiving funding. Conversely, they need to avoid crowding-out commercial and local funding sources (including client deposits).

Additionality is a principle that needs attention at all stages of the project/investment cycle. The strategy and operational guidelines set the stage by defining the rationale for a funder’s interventions and by describing how the different instruments should be used to add value. Funders can include additionality checks in their due diligence or in project/investment approval processes. Guidelines on how to structure and price instruments can help staff adapt interventions to market conditions. For example, cost-sharing guidelines and market-based pricing methods can limit the risks of crowding out. Funders should have a clear vision of when and how to exit responsibly from a funding relationship with an institution or from a given market to avoid dependence on development assistance. At any time in the project/investment cycle, a funder should be able to articulate how its interventions lead to sustainable change and, ideally, measure progress against this vision.

While a range of instruments is needed to build inclusive financial markets, not every funder needs to have a broad palette of instruments. The suitability of the instrument(s) for the activities supported by a funder and the way they are used in a given market is what matters, not the number of instruments.

Governments are natural partners when funders work on the policy environment and the regulatory and supervisory framework. However, for work on other levels, and especially the retail level, it is harder to build the capacity of retail financial service providers if funders must channel all or almost all support through government bodies. Funders whose primary instrument is large loans to governments often have a comparative disadvantage with regard to supporting retail financial service providers. Experience has shown that governments are usually not suited to directly manage wholesale lending projects. However, exceptions do exist where governments select intermediary bodies that have the systems and technical skills to appraise retail institutions, provide implementation support, and track performance.

Funders that are able to work with a range of private actors are well-suited to support the retail (micro) and market infrastructure (meso) levels. Working effectively with private actors requires internal processes that are sufficiently flexible and efficient.

Channeling funding through components of multisector projects can undermine performance. If the financial inclusion component represents only a small part of a multisector project, it is unlikely to receive the technical attention needed. A perceived trade-off between supporting sustainable financial services and meeting specific objectives for a target group may lead funders to neglect sustainability. As a result, components often decapitalize quickly due to high costs, subsidized pricing that does not cover costs, and poor collection rates. Without sustainability, services may cease when a project is completed.

SmartAid looks for the following:

•   Funders that have instruments that fit their strategy for financial inclusion and give them the means to appropriately fulfill their strategy.

•   Funders that consider and analyze the additionality of their projects and investments at project design and when reviewing their portfolio.

•   Funders that define responsible exit strategies when designing projects and investments.

•   For debt: Funders that offer loans in local currency and price loans responsibly and in line with local market conditions.

•   For grants: Funders that use grants for time-bound interventions with longer-term benefits for the institution and the market. The use of grants should be transparent (e.g., not hidden in concessional loan terms) and performance-based. Funders should give grants only after assessing the grantee’s capacity to sustain project benefits beyond the duration of the grant and use cost-sharing mechanisms to enhance ownership.

•   For loans to government: Limited use of loans to government to fund retail financial services: Little financial inclusion programming through credit components of larger, nonfinancial sector projects, or systems to ensure technical oversight if used regularly.

Documents to be provided as evidence:

Table of instruments with terms and conditions (see template provided by CGAP).

VII. Resources

CGAP. 2012. “Portfolio Reviews: Resource Guide for Funders.” Technical Guide. Washington, D.C.: CGAP.

———. 2010. “Performance-Based Agreements: Incorporating Performance-Based Elements into Standard Loan and Grant Agreements.” Technical Guide. Washington, D.C.: CGAP.

———. 2009a. “Improving Effectiveness from Within: SmartAid for Microfinance Index.” Brief. Washington, D.C.: CGAP.

———. 2009b. “Measuring Results of Microfinance Institutions—Minimum Indicators That Donors and Investors Should Track.” Technical Guide. Washington, D.C.: CGAP.

———. 2006. “Good Practice Guidelines for Funders of Microfinance.” Washington, D.C.: CGAP.

Annex I: Terms of Use of the SmartAid Index

Funders using the SmartAid Index to assess the effectiveness of their internal systems agree with the following terms of use. Contact CGAP for further information or questions.

Terms of use of the SmartAid Index

•   The SmartAid Index is useful only for agencies with a motivation to improve their effectiveness, and that are eager to learn from the experience of others.

•   The name and logo of CGAP can be used only with the explicit agreement of CGAP.

•   Users are expected to follow the steps indicated in this technical guide.

•   Users must always use the most recent version of the SmartAid toolkit.

•   In return for the use of the SmartAid toolkit, users must agree to send the completed questionnaire to CGAP. This step is necessary to receive the score.

•   Validated scores are entered in CGAP’s database, which allows funding agencies to compare their results with those of their peers.

•   All validated reports are published on the CGAP website, except if the management of a participating organization formally requests CGAP not to publish the report.

•   Users agree to communicate any suggestions for improvement of the SmartAid Index to CGAP and to participate in feedback surveys or evaluations on the SmartAid Index.

•   CGAP reserves the right to perform a quality check of the questionnaire responses, staff survey, and documents before publication or dissemination of any report.

Annex II: Compact for Better Aid for Access to Finance

1 The generic term “project” is used throughout this Technical Guide to describe an individual intervention supported by a funder, and it is typically assigned a unique project identification (ID) number. Terminology varies across different organizations, and sometimes the terms “investment” or “program” are used synonymously with “project”.

2 See Annex II.

3 See what funders say about SmartAid in this short video: http://www.cgap.org/photos-videos/smartaid.

4 The online survey is supported by www.surveymonkey.com.