Tool 7: Thematic Reviews

Thematic reviews are in-depth analyses of limited topics—or themes—which can provide invaluable insights for market conduct supervisors (MCSs). Various countries sometimes refer to thematic reviews as horizontal analyses, thematic analyses, industry reviews, cross-sector reviews, cross-firm reviews, and market studies.

The most common objective of a thematic review is to identify and investigate one or more consumer risk(s) across a market or subsector, including how varying practices within financial services providers (FSPs) may impact a risk. A thematic review may also be used to check compliance or measure the impact of specific aspects of a recently issued regulation, or to learn more about a new topic, product, or provider type. Although the results of a thematic review may be the basis for institution-focused follow-up work, the focus of a thematic review is on the market as a whole or a subsector of a regulated sector. A thematic review can be used to address unfair consumer treatment or to focus attention on vulnerable segments.

Benefits and opportunities

• Flexibility. The design of each thematic review is inherently flexible.
• Depth. A well-planned thematic review allows an MCS to obtain deep insights on a specific topic that has raised supervisory concern or interest.
• Comprehensiveness. A well-designed thematic review can generate powerful cross-sector insights from a range of FSPs, including comparisons of conduct, identification of outliers, and analyses of different consumer risks or outcomes by various types of FSPs.
• Supervisory effectiveness. This tool supports a risk-based supervisory approach and can help an MCS optimize the use of resources by sharpening the focus of supervisory planning. Depending on the techniques used, this tool can be a cost-effective way to build deep supervisory expertise on priority issues.
• Segmentation. It can help an MCS to become more customer-centric by enabling a greater focus on vulnerable consumer segments.
• Proactivity. This tool gives an MCS the opportunity to identify and understand good and bad FSP conduct practices, including those related to emerging issues that are not necessarily detailed within a regulation. Based on such knowledge, the MCS may use moral suasion with individual FSPs to push for best practices without imposing them through regulation. 
• Feedback. The insights from a thematic review may be used for institution-focused follow-up or to support regulatory reform.
• Dissemination. Public dissemination of a thematic review report increases accountability and can provide additional resources that allow consumer groups and the media to push for better practices by FSPs.

Back to top ↑

Characteristics of this tool

A thematic review typically uses a range of techniques to focus on in-depth analyses of a specific topic or theme across a market or subsector. The topic of study can be virtually anything, for example:

• A specific consumer risk, such as over-indebtedness or gender and other types of discrimination in credit markets.
• A specific compliance issue, such as an FSP’s compliance with mandatory disclosure of information about its complaints handling channels and procedures.
• An FSP practice or process, such as disclosure of key terms at time of sale, debt collection, or online disclosures.
• A review of standard-form consumer contracts for specific types of products.
• A new product (e.g., virtual currency), including how it works and is used by consumers.
• A review of how FSPs deal with vulnerable consumers (e.g., low-income women, elderly individuals, smallholder farmers).

Following are examples of MCSs that use the thematic review process and the focus of some of those reviews.

Australia

The Australian Securities and Investments Commission (ASIC) uses thematic reviews as an integral element in its supervisory toolkit. ASIC has conducted numerous reviews of a range of issues. For example, a review of claims procedures in total and permanent disability insurance revealed negative consumer outcomes from current industry practices, such as strict definitions used by insurers that resulted in denial of 60 percent of claims.

Canada

The Financial Consumer Agency of Canada (FCAC) regularly conducts thematic reviews. A recent review which looked at how banks handle consumer complaints found several weaknesses that were subject to supervisory follow-up. The weaknesses encouraged FCAC to issue further guidance on its supervisory expectations for complaints handling.

Ireland

According to its annual supervisory planning, the Central Bank of Ireland regularly conducts thematic reviews on consumer protection issues. Rather than reviewing reports, it typically publishes letters to the industry that summarize its findings. A recent example is the central bank’s review of diversity and inclusion across a sample of insurance firms, including analyses of remuneration by gender and analyses of diversity and inclusion policies, procedures, practices, and monitoring.

Malaysia

The Securities Commission of Malaysia regularly uses thematic reviews. In 2020, it conducted 416 institution-focused supervisory assessments and three thematic reviews involving 282 intermediaries on identified areas of risk. One review covered compliance with anti-corruption policies and procedures, which led the Commission to publish key findings and good practices obtained from the review; send communications to senior managers to share key findings about their FSPs; and issue supervisory letters against individual FSPs that needed to significantly improve their policies, procedures, risk assessment, monitoring, and training.

South Africa

The Financial Sector Conduct Authority (FSCA) includes thematic reviews of cross-cutting emerging risks as a key element of its conduct supervision toolkit. FSCA and its predecessor, the Financial Services Board, have undertaken thematic reviews on topics such as the sale of add-on auto insurance during car sales (2020) and insurers’ complaints management practices (2014). The latter review supported the development of a new regulation that aligned with customer outcomes. FSCA aims to publish aggregate findings from thematic reviews for the industry and use its insights toward regulatory and supervisory reform.

Back to top ↑

How to use this tool

Thematic reviews may employ a variety of specific techniques or tools (including others described in this toolkit) to analyze many data types and sources. For example, a thematic review may include offsite analysis of granular datasets exclusively requested for a review and not usually collected through regulatory reporting, an onsite visit to an FSP, remote interactions with an FSP, in-person or remote consumer surveys, consumer interviews, and mystery shopping. MCSs choose which tools to use according to the task at hand and available resources. Some thematic reviews are quick and simple exercises while others are resource intensive and may require a significant time commitment.

There is no single recipe for a thematic review. In fact, an MCS may take a completely different approach to reviewing the same topic at a different time. This lack of standards for thematic reviews can be illustrated by the different ways two authorities reviewed complaints handling at FSPs:

UK’s FCA

A 2014 thematic review covered complaints handling by non-deposit-taking mortgage lenders and third-party mortgage administrators. The review focused on self-assessment questionnaires applied to a sample of 15 retail FSPs and five trade bodies, as well as information and documentation requests for offsite review. The FCA sought input from the Financial Ombudsman Service and consumer bodies. It also organized a working group with the FSPs and organizations to discuss complaints handling issues.

Canada’s FCAC

In 2020, the FCAC conducted a thematic review of complaints handling at regulated banks in response to a request from the Minister of Finance. The review took approximately seven months and included the analysis of FSP documentation such as complaints files and interviews with bank employees. FCAC additionally commissioned consumer research of 5,000 bank clients.

Regardless of variations across authorities, the decision to conduct a thematic review follows a logical process. CGAP organizes that logic into the following nine steps:

STEP 1:

DEFINE AND JUSTIFY THE TOPIC

A thematic review can be resource intensive. It is important to clearly justify its need and purpose at the outset. For example, based on the issuance of a new regulation on minimum standards of internal complaints handling by FSPs, an MCS may want to learn how well FSPs handle consumer complaints. Alternatively, the MCS may be triggered by a suspected risk, such as misconduct based on a sharp increase in the number of complaints related to a specific issue. The following questions may help an MCS define its thematic review topic:

• What is leading you to choose a specific topic?
• Why is that topic more important than others?
• What do you expect to achieve through the thematic review?

STEP 2:

DEFINE THE RESEARCH QUESTIONS

If an MCS chooses the topic of complaints handling, for example, the thematic review could answer specific questions, such as:

• Are the channels and procedures offered to consumers adequate for vulnerable client segments to use as well?
• Are the criteria FSPs use to reject or accept a complaint fair and clear?
• Is the complaints handling function at FSPs independent from commercial areas?
• Do FSPs use adequate complaints management information systems?
• Do complaints handling units use effective quality controls to evaluate customer interactions? 
• Is performance evaluation of the complaints handling function effective? 
• Are efforts in place to reduce complaints resolution time? Are they adequate?
• Have consumers had positive experiences in resolving complaints with FSPs?
• Are there indications of bias or discrimination that could hurt women and LGBT+ consumers, plus other minority or vulnerable groups (e.g., differences in resolution time and outcomes, complaints rejection)?

The importance of designing clear and focused research questions to get strong results out of a thematic review cannot be emphasized enough. Imprecise or ambiguous questions may lead to weakness and a lack of focus in the data gathering and analysis phases.

STEP 3:

DEFINE THE TOOLS

Once an MCS has identified specific research questions, it is easier to identify which tools will be the most useful. Following the same example of a thematic review of complaints handling at FSPs, the tools could include:

• Analysis of aggregated complaints data regularly reported by FSPs.
• Analysis of aggregated complaints data collected by financial sector ombuds.
• Analysis of granular complaints datasets requested by FSPs.
• Analysis of disclosure of complaints handling procedures on FSP websites.
• Visits to complaints handling units at FSPs and interviews with managers and operational staff.
• A consumer survey. 
• In-depth consumer interviews with actual complainants identified through granular data analysis.

STEP 4:

DEFINE THE SAMPLE

Having identified a wish list of the tools an MCS wants to use, it is time to consider how many institutions it is able to cover, plus the time and number of staff members required. For example, an MCS may discover that it does not have sufficient budget to perform a consumer survey or cannot cover as many FSPs as anticipated. If there are too many FSPs to cover, an MCS may need to define a sample for each sector (e.g., the banking sector) based on clear criteria. It may choose to include in that sample at least one FSP from each peer group (e.g., large, medium, small banks) or to prioritize FSPs with a higher risk profile, according to the MCS’ risk rating methodology.

STEP 5:

GATHER DATA

Gather all documentation, data, and other information from FSPs prior to moving on to onsite visits or other in-person interactions with FSPs and consumers. While still in an office setting an MCS may need to follow up with FSPs to seek clarifications or to request additional data.

STEP 6:

CONDUCT THE ANALYSIS

It may take a few days or months to conduct a thematic review, depending on the topic and the complexity and depth of analyses. For a thematic review on complaints handling mechanisms at FSPs (particularly with a large number of FSPs), the wide range of questions and tools used would probably necessitate a few months and at least two supervisors to conduct analyses and coordinate consumer surveys and interviews (completed by a research company).

STEP 7:

PRODUCE THE REPORT

A thematic review report is very different from an institution-focused inspection report. It looks and feels more like a research report (see examples under Characteristics of this tool). An MCS may produce an internal report that names specific FSPs, separate from the report it creates for public consumption.

STEP 8:

DISSEMINATE FINDINGS

After putting together a thematic review report, it is useful to organize at least one engagement session with all FSPs (beyond those in the sample). The engagement usually focuses on describing the findings in terms of good and bad practices observed. Even without explicitly naming and shaming, in smaller markets where FSPs know how their competitors operate, such gatherings often result in implicit naming and shaming. Discussions should not be conscribed to regulatory compliance only. An MCS can use the opportunity to gather feedback from FSPs on regulatory vagueness and clarify supervisory expectations—which can go beyond a regulatory text to highlight best practices in complying with a regulation. It may be useful to disseminate clear findings to consumer associations, dispute resolution mechanisms, journalists, and other stakeholders that would benefit from more detailed explanations than what is included in reports produced for public dissemination. Direct dialogue with an MCS often helps stakeholders play a more active role in improving market conduct.

STEP 9:

FOLLOW UP

An MCS may need to follow up with individual FSPs either by producing FSP-specific reports to be shared with individual FSPs or by writing a letter to elicit follow-up actions by FSPs.

Back to top ↑

Limitations of this tool

• Resource intensiveness. Depending on how it is designed and which techniques are used, a thematic review may be resource intensive. MCSs need to find a balance between comprehensive and superficial thematic reviews. The choice depends on the gravity of the issues and available resources.
• Unrepresentativeness. In large markets, an MCS is unlikely to cover all FSPs of a certain type or across different types, resulting in a partial cross-sector analysis.
• Data quality. It is important to have a method for designing good samples. An inadequate sample may lead to distorted results and produce an inaccurate view of best and worst practices in a market or subsector.

Back to top ↑

Other resources

• Market Conduct Supervision of Financial Services Providers: A Risk-Based Supervision Framework (AFI 2016)
• Sex-Disaggregated Data Toolkit (AFI 2017)
• Thematic Assessment of Diversity and Inclusion in Insurance Firms (Central Bank of Ireland 2020)
• Implementing Consumer Protection: A Technical Guide (CGAP 2013)
• Financial Consumer Protection: 3 Steps to Better Customer Outcomes (CGAP 2020)
• DFS Supervision Toolkit—Collection (CGAP)
• Elevating the Collective Consumer Voice in Financial Regulation (CGAP 2021)
• Supervisory Toolbox (FinCoNet)
• High-level Principles on Financial Consumer Protection (G20/OECD 2011)
• Collecting and Using Gender-Disaggregated Data for Financial Policymaking (UNSGSA 2020) 
• Good Practices for Financial Consumer Protection (World Bank 2017)

Back to top ↑