FCDO Comment on CCG Management Response to Evaluation of FCDO’s Climate Compatible Growth (CCG) Programme Phase 1 Report (May 2024)

Several drafts of CCG’s Management Response to the Independent Evaluation Phase 1 Report have been reviewed by the FCDO. Detailed comments have been made and addressed by CCG during an iterative drafting process.

The FCDO staff overseeing the CCG Programme are now fully content with this CCG Management Response. We endorse the actions proposed and are overseeing their implementation through our ongoing oversight of the programme.

Evaluation of FCDO’s Climate Compatible Growth (CCG) Programme Phase 1 Report (May 2024)

CCG MANAGEMENT RESPONSE: CONTEXTUAL INTRODUCTION

CCG welcomes and accepts or, in some instances partially accepts, the recommendations of the external evaluation and provides the justifications in so doing. This evaluation was conducted at the request of the FCDO to inform early learning and strengthen the CCG programme’s strategic direction. The evaluation report came at a pivotal time, just prior to CCG programme being granted extension through to March 2030 and in the context of an increasingly complex climate-compatible growth ecosystem striving towards the Sustainable Development Goals era. We appreciate the acknowledgment of CCG having made substantive progress in its relatively short life since 2020, notably its advanced Country Partnerships programmes, its growing International Partnerships collaborations, its commitment to open-source data and models, and its overall ambitious strategy to push academic boundaries through its research.

This management response was developed through a consultative process that engaged the CCG Directorate and the PI senior leadership. Although the senior management and administrative staff at CCG have been bolstered with additional content leadership and administrative capacity since the Phase 1 Evaluation, CCG is currently monitoring and assessing whether this new capacity is sufficient to fully support the programme portfolio and maximise its overall effectiveness. CCG acknowledges the requirement for establishing areas of responsibility/seniority within the CCG structure and hierarchy and has set up a formal structure to better communicate roles and responsibilities as well as formalised the representative voting membership within the Directorate.

Additionally, the programme is undergoing a comprehensive review of its research operations, portfolio structure, and International and Country Partnerships interfaces, with an emphasis on enhancing collaboration across workstreams. The issue of insufficient collaboration between workstreams, such as those focused on modelling, is being addressed through a significant Programmed Research Strategy reboot in 2024. This reboot seeks to develop a new integration proposal and strategy by the end of the year, which will sharpen alignment and synergy across all workstreams. A follow-up reboot is planned for 2027, with annual updates to those reboots in-between. In terms of International Partnerships Activities (IPAs), of which there were several, CCG is streamlining its approach by consolidating these activities into three distinct units, each guided by a clear strategy and operational plan.

The sharpened Country and International Partnerships strategies are designed to better align partner efforts with CCG’s objectives. A coordination process has been established to assess Programmed Research strategies through the lens of − and by linking them more directly to − outcomes, both at the country and international levels. This also provides a framework for diagnosing and informing funding allocations. Regarding the sustainability of Partner Country activities and the future of the related Special Interest Groups (SIGs), a strategic approach is being adopted. This includes a process for ensuring that CCG knowledge continues to be used in countries beyond the lifetime of the CCG programme. A lighter mode of engagement, through direct “engagement-research” coordinated by International Partnerships, is also being introduced for Affiliate Countries, offering targeted interventions with specific outputs, requiring limited resources. Finally, a review of the “Southern Partner Fund” (SPF), “Flexible Research Fund” (FRF), and Engagement Fund is in progress. This process includes piloting new operational models with a strong emphasis on transparency and reporting to ensure that flexible funding allocations remain appropriate and effective in meeting CCG’s goals. CCG has recently piloted the launch of the Engagement Fund to address some of the country programme funding gaps, in addition to the flexible and southern partner funding modalities. However, we recognise that the Engagement Fund will not fill all funding gaps and CCG work will endeavour to deliver within the resources available while efforts continue to attract other funding sources.

CCG continues to be constrained by a 12-month budget cycle. This is due to several consecutive one-year HMG Spending Review Periods (SRPs) necessitated by economic uncertainty caused by the COVID-19 Pandemic and Brexit. CCG have communicated their related concerns to FCDO who have been very understanding and endeavour to support CCG’s recommendation for the extension of budget to revert to the previous three-year cycles; however, this is outside the control of CCG. It is also outside the control of the FCDO as SRPs are set by HM Treasury.

The Oxford Policy Management (OPM) evaluation team could not fully evaluate work that was in its early inception at the time of the phase 1 evaluation, such as the development of the Partnerships and Programmed Research strategies, the development of the Gender Equality and Social Inclusion (GESI) strategy, and cross-cutting initiatives such as Data-to-Deal (D2D) and critical minerals which aim to strengthen interlinkages across Partnerships and research workstreams. In particular, D2D is divided into integration activities between workstreams to ensure programme alignment; an implementation strategy to ensure uptake beyond other joint CCG activities; and activities that investigate the political economy realities of D2D deployment as well as constructive criticism of its process.

Aligned with its strategic approach of lessons learnt in its first phase, there are ongoing efforts to renew and make modest changes to CCG’s Theory of Change (ToC) and LogFrame. CCG continues to focus on reporting of outputs as well as projection of potential outcomes as it transitions from its earlier phase into operationalisation, which is further informed by the external evaluation.

CCG, in parallel to maturing its programme to focus on delivering outcomes, aims in its subsequent phase to also develop an exit strategy which considers its legacy beyond 2030.

The next section tabulates (Table 1) CCG management responses to the twenty-four recommendations of the CCG external evaluation report, followed by other key evaluation findings (Table 2) referred to in the main body of the Phase 1 report, which are also acknowledged.

ABBREVIATIONS

ACR	Annual Consortium Review (CCG)
ADB	Asian Development Bank
AEFI	Africa Energy Futures Initiative
AG	Advisory Group (CCG)
AGA	Accountable Grant Agreement (FCDO)
AISESA	African Institute for Sustainable Energy and Systems Analysis
CCG	Climate Compatible Growth
CGE	Centre for Global Equality
CP	Country Partnerships (CCG)
D2D	Data-to-Deal
EMC	Energy Modelling Community
EMP	Energy Modelling Platform
EOI	Expressions of Interest
FCDO	Foreign, Commonwealth and Development Office
FRF	Flexible Research Fund (CCG)
FY	Financial Year
GEAPP	Global Energy Alliance for People and Planet
GESI	Gender Equality and Social Inclusion
HIC	High-Income Country
IAEA	International Atomic Energy Agency
IEA	International Energy Agency
IP	International Partnerships (CCG)
IPA	International Partnerships Activity
IRENA	International Renewable Energy Agency
LMIC	Low- and Middle-Income Country
MEL	Monitoring, Evaluation, and Learning
OIF	Operational Intelligence Framework (CCG)
OLC	OpenLearn Create
OPM	Oxford Policy Management
OPSIS	Oxford Programme for Sustainable Infrastructure Systems
PI	Principle Investigator
PML	Programmed Research Leadership (CCG)
PR	Programmed Research (CCG)
RD	Research Directorate (CCG)
SDK	Starter Data Kit
SIG	Special Interest Group (SIG)
SPF	Southern Partner Fund (CCG)
SRP	Spending Review Period
ToC	Theory of Change
ToR	Terms of Reference
UNESCO	United Nations Educational, Scientific and Cultural Organization
UNOPS	United Nations Office for Project Services
VfM	Value for Money

TABLE 1 - CCG Evaluation Phase 1 Recommendations
#	Recommendations	Priority	CCG Lead
Clarity of intervention logic:
1	CCG should consider further developing the ToC to clarify what outcome is expected from the Open Learn platform and Modelling summer schools beyond ‘CCG countries’. Response (pls select): Partially accepted This recommendation appears to be based on a misunderstanding of how OpenLearn (OLC)/Energy Modelling Platform (EMP) fit into the Theory of Change (ToC), which is mainly a question of improved communications and clarification.	low	Operational Intelligence Framework (OIF)/ International Partnerships
CCG Management Response/Strategic Action Plan	Timeline
1.1 The current ToC includes explicit mention of the uptake of CCG knowledge products. These include the Open Learn Create / Energy Modelling Platform (OLC/EMP) activities, which service countries in CCG target regions as well as Partner Countries. There is no contradiction, nor need to update the ToC. In CCG Partner Countries, the entire ToC can be applied in a comprehensive and holistic manner. In other countries, this is evidently not possible. However, their engagement with CCG capacity-building services allows them to participate in more targeted elements of the ToC. 1.2 Changes to the LogFrame have been proposed for 2025/26 (pending FCDO discussion) to better incorporate the outputs of the modelling summer schools (that is, the EMPs). These changes include the addition of new indicators and sub-indicators for OA 1.4 Capacity building events. Specifically substituting the main indicator with 'number of capacity-building events', introducing a sub-indicator for the 'number of participants per capacity-building event', and converting the 'percentage of participants who would recommend the event to peers' into a sub-indicator. 1.3. Specific actions that have been taken, and are ongoing, include the development of a community of practice for EMP graduates and trainers that goes beyond CCG Partner Countries to include CCG’s target regions. This is set up to attempt to sustain momentum and crowd-in resources, including in-kind community support to enable outcomes with relatively low input from CCG core resources. For more targeted intervention and to glean in-kind contributions, two new categories of countries have been developed. These include ‘Affiliate Countries’. These are countries in target regions who are not Partner Countries. These countries are embedding CCG outputs that are evolving into outcomes, often with locally led, multi-stakeholder partnerships (using CCG knowledge products) being resourced by international partners. The other category is ‘Demonstrator Country’. These are countries that are not in CCG target regions but co-create and use CCG knowledge products that result in demonstrable outputs, outcomes, and impact. These can then be translated to apply in CCG target regions, and to market the potential of outputs, outcomes, and impacts to be achieved. 1.4 In particular, whenever CCG capacity-building services are delivered, the LogFrame output of ‘Enhanced individual capacities to use decision support tools, processes and policies’ will result, as will ‘Enhanced southern individual research capacities’. These capacity outputs are expected to feed through into outcomes in a variety of ways; albeit that such outcomes are more challenging to document in the case of non-CCG Partner Countries, because they are more highly dispersed. First, CCG capacity building may become ‘embedded in international organisations’. We have specific examples of this with the International Energy Agency (IEA) adopting OLC/EMP as their default capacity-building route for their client countries in Africa. Another example is how the International Atomic Energy Agency (IAEA) and International Renewable Energy Agency (IRENA) have partnered with CCG to amplify capacity-building efforts around modelling tools that they have developed (MAED and FinPlan in the case of IAEA and FlexTool in the case of IRENA). Second, CCG capacity building may become ‘embedded in participating countries’. This is best documented in the case of universities, where several African universities (in advance of any formal Flatpack Programme, a ready-developed set of teaching units for use in academic settings) spontaneously integrated CCG modelling training materials into their own curricula. This is known to have taken place in Uganda and the Democratic Republic of the Congo, for instance. There are also examples of EMP graduates going on to apply modelling skills in informing in-country policymaking, as was the case of the Rural Electrification Masterplan in Tanzania. CCG management is in the process of collecting more information on outcomes generated by OLC/EMP graduates. Third, OLC/EMP graduates can also go on to produce research papers that make use of the analytical skills acquired. In some cases, this is encouraged and supported via the use of journal special issues.	ongoing
2	If the ‘Data-to-Deal’ (D2D) framework is to inform CCG country strategies it needs to be further developed to account for how political economy issues play a part in investment decisions, and then communicated across CCG in a consistent manner. Response: Accepted	Priority: low	Partnership Director/ International Partnerships
CCG Management Response/Strategic Action Plan	Timeline
This recommendation is accepted, and the following actions have been taken by management. 2.1. Internally communicating the 'Data-to-Deal' (D2D) framework while considering the political economy issue. It should be noted that the Political Economy Unit was only established after the CCG external evaluation had been conducted, which will be part of these efforts. (This unit will conduct country, sector, and project-specific political economy analysis as needed, support policy engagement and identify research synergies across workstreams). A full participatory session on D2D was held at CCG’s General Meeting in April 2024 to socialise the concept. This session involved not only presenting the framework and discussing it, but also emphasising that for the D2D framework to inform CCG country strategies, it needs further development to incorporate how political economy issues influence investment decisions. 2.2. Developing a detailed Terms of Reference (ToR) for a D2D engagement collaboratively between workstreams (International Partnerships, Country Partnerships, and Programmed Research) to ensure a shared understanding and facilitate integrated efforts. This shall entail providing much greater operational clarity regarding what ‘Data-to-Deal' means and how the approach would be implemented at the country level, including stakeholder facilitation, analytical advances, and capacity-building aspects, alongside technical elements and analytical products. 2.3. Explicitly incorporating the role for Political Economy Analysis as part of Step One of the D2D process (High Level Political Vision). Although this is currently work-in-progress, this shall likely include consideration of contextual insights of political and economic landscapes, stakeholder mapping, potential political risk assessment where warranted, strategic alignment, and advisory services to inform political and economic context.	ongoing
Focus of research:
3	CCG should consider whether the balance between demand-led and more academic research is appropriate across each of the Programmed Research workstreams (and what the balance says about the relevance of the research). This, along with the investment splits referenced in recommendation 3 above could be reviewed as part of the annual planning process and resource allocation process. Response: Accepted [However, note that cross reference to the draft recommendation #3 presumably referring to ‘the ideal split’ is not applicable as reference to split was subsequently removed from the evaluation report]	medium	Research Director
CCG Management Response/Strategic Action Plan	Timeline
3.1 There is a recognition that a balance is needed between demand-led and more academic focused research. The goal is to take forward these different research types together or independently, to ensure that the different outcomes in the ToC can be effectively met. The Research Directorate (RD), working with the Programmed Research Leadership (PRL), is developing the Research Strategy to ensure that a balance is maintained, as described below, through careful articulation of research strategies. This will include both gathering of demands via Country and International Partnerships, and via researchers’ own links with in-country partners. 3.2 The PRL, which consists of all research investigators (i.e. all with formal academic positions in their respective organisations, as opposed to other researcher roles) in CCG, is currently revising the existing research workstream structures based on a review which commenced during its strategy meeting held on 19 June 2024. The proposed revisions aim to ensure that longer and deeper research is carried out alongside demand-led activities, such as green hydrogen, critical minerals, and artificial intelligence, while also fostering integration across workstreams. The process is currently being defined with the RD and will likely include internal OIF review, sign-off by CCG leadership, and consultations with the FCDO and the Advisory Board. 3.3 Any proposed new structure changes, if applicable and subject to approval due process, will be explained in the accompanying CCG overarching Research Strategy – which will help define the balance between sectors and methodological approaches – and sub-strategies (workstreams / cross-cutting-themes). This is set to be completed by the end of the year (2024), incorporating comments from paid-for external peer review and FCDO by April 2025. 3.4 Based on the CCG’s overarching Research Strategy and considering the revised research workstream structures, workstream projects will be defined by April 2025. The overall Research Strategy, and accompanying project plan, will be key in shaping research direction, and will also be informed by country strategies, to maintain the balance between demand-led and (as noted in the evaluation) more ‘academic boundary-pushing ‘blue sky’ research’.	April 2025
4	CCG should consider whether its allocation of funds to the SPF, FRF and the new Engagement Fund planned for FY 24/25 are sufficient to allow workstreams to engage more with LMIC researchers and for LMIC researchers and research institutions to be engaged on / lead research at a level commensurate with the intentions of the programme. This may involve FCDO and CCG management agreeing a clearer picture of what constitutes a good level of LMIC involvement in research activity in the context of CCG. It may also involve consideration of whether the assessment process for proposals, in particular the consideration of co-financing offered by applicants in the scoring proposed for the engagement fund, potentially disadvantages LMIC institutions. This recommendation should be viewed in conjunction with recommendation 18. Response: Partially accepted Subject to FCDO and CCG agreement [to unfold] on what constitutes a good level of Low- and Middle-Income Country (LMIC) involvement and within the parameters of available resources.	high	Directorate
CCG Management Response/Strategic Action Plan	Timeline
FCDO and CCG management have yet to agree what constitutes a good level of LMIC involvement in the research activities through to 2030. This process will also be informed by the application of the SPF and Engagement Fund, as well as be informed by the Research and Country Partnerships strategies relevant to LMIC involvement, and this will unfold by the end of the year. 4.1 CCG is in the process of developing a revised strategy for the SPF and FRF. This began with a coordination effort led by the Operational Intelligence Framework (OIF) in 2023. As with other ‘projects and outputs’ in CCG, flexible fund proposals are scored against CCG’s ToC, LogFrame, and associated directive documents. Southern Partner Fund (SPF) 4.2 The Southern Partner Coordination team, which consists of representatives from Programmed Research, the OIF Unit, Partnerships, and Loughborough University management, were scheduled to meet in October for a full-day workshop in order to propose a revised strategy for SPF going into CCG Phase 2. Multiple options will be considered, reflecting on what has/has not worked so far. 4.3 In the same meeting, we will discuss the level of resources needed to best establish long-term partnerships and build capacity on the ground. If needed, a request for a larger funding allocation will be made. 4.4. Following this, the recommendations will be presented to CCG leadership and the wider CCG team for internal review, and also shared with the FCDO for a consultative process at its subsequent workshop opportunity. 4.5 The proposed changes will be workshopped with the FCDO to understand the palatability of the proposals that are developed. Thereafter it will additionally undergo a process of external scrutiny with CCG country representatives. 4.6 Thereafter final signoff will be sought from the CCG management, and then the FCDO. The final strategy for SPF is envisaged to be ready by the end of December 2024 so that funding requirements can be accounted for and process documents drawn up before the start of CCG Phase 2. Flexible Research Fund (FRF) >>> Refer to recommendation 14 for more details. 4.7 The FRF Coordination team, which consists of representatives from Programmed Research, the OIF Unit, Partnerships, and Loughborough University management, will meet in October for a full-day workshop to draw up a strategy for FRF going into CCG Phase 2. It will go through similar engagement with CCG, FCDO, and external partners before sign-off. Some issues we will address include: (i) the quality of deliverables not meeting expectations; (ii) the workload on workstream leads being higher than expected or planned; and (iii) the call attracting institutions from the Global South, which was not necessarily the original intention. 4.8 In response to some of the concerns raised, we have already made the following improvements to the FRF. For the 2024 FRF calls, requests for Expressions of Interest were included as an additional step. This has enabled us to pre-filter applications, working towards consistency in assessment of the full proposals. 4.9 Revised evaluation criteria will be used to ensure proposals are appropriately assessed against clear and transparent standards. Guidelines for the reviewers have been drawn up to ensure greater consistency in the quality and detail of the reviews. 4.10 As the FRF is competitively tendered, it is essential to maintain the principle of free and fair competition throughout the process. Adjustments will be made to ensure this is the case, and they will be reflected in the revised documentation listed in 4.9. Engagement Fund 4.11 The Engagement Fund is undergoing a trial period in FY5 (this is the first FY it has been offered in CCG) to collate learnings from the research funds as well as processes of contracting / implementing. 4.12 A lessons learnt and full process document for the Engagement Fund will be finished by March 2025. This will include consideration with respect to monitoring its efficacy and scoring, ensuring that projects eligible for Engagement Fund disbursement are fairly chosen. Flexible Funding 4.13 The new Standard Operating Procedure for the SPF, FRF, Engagement Fund will be inserted in to the CCG Project Manual and subject to its annual monitoring, evaluation, and learning (MEL) activities. While various internal and external groups will provide input on process and proposal scoring, internal scoring will take place as a matter of course by the OIF to help ensure flexible funds and programme-wide performance is consistent.	March 2025
5	FCDO and CCG should consider whether plans to expand to more countries risks undermining the ability to resource in-country demand, particularly for modelling support, or the investment at the level needed to build sustainable national research capabilities in existing country programmes. Response: Partially accepted FCDO requested for the CCG Country Partnerships to complement up to a portfolio of 8 countries (this process was already initiated and ongoing before the evaluation was completed) and was based on a greater value for money perspective.	high	Country Partnerships
CCG Management Response/Strategic Action Plan	Timeline
5.1 FCDO and CCG agreed this was a concern and a risk, and therefore agreed, upon discussion with relevant stakeholders, that eight Partner Countries would be the maximum Country Partnerships portfolio. Given this, two new Partner Countries (Nepal and Malawi) were selected in June 2024 to achieve eight Partner Countries. The CCG Partnerships Directorate created a shortlist of countries that overlapped with FCDO target countries and CCG countries of interest based on their energy and climate ambitions to date. We then met with the different FCDO Offices that are based in those countries to explore how CCG could complement the activities already ongoing within the shortlisted countries. Based on CCG’s own desktop research and the communication with the local FCDO Offices, Nepal (South Asia) and Malawi (Southern Africa) were chosen to be added into the Country Partnerships’ team portfolio. 5.2 Drafts of six Country Partnerships strategies will be completed by the end of September 2024. 5.3 A two-page engagement plan will be shared with Post in the two new Partner Countries (Nepal and Malawi). 5.4 Political economy analysis, as well as an initial mission with take place in the two new Partner Countries. 5.5 In October 2023, FCDO made the following remark over email about why the Country Partnerships (f.k.a. National Partnerships) portfolio will not extend beyond eight Partner Countries: The decision has been taken that CCG will not scale up to more than eight partnerships in the coming years for the following reasons: CCG partnerships draw upon the academics within CCG’s research consortium for their delivery, via: direct engagement on training and capacity building; delivery of research outputs that respond to demand via the Engagement Fund; delivery of the Flexible Research Fund commissioning process and support to the selected delivery partners; research consortium support to the SIGs. It is not possible to rapidly scale up the number of academics within the CCG research consortium in order to support a larger number of Country Partnerships. This is because CCG academics are experts in their field; it is not easy to identify and recruit people with similar levels of expertise, and because university recruitment procedures do not facilitate rapid recruitment of a large number of additional staff. CCG is perceived at present as delivering a good balance for a Research and Development programme between: (1) original, academic-led research (which forms the foundation of the partnerships); and (2) uptake and capacity-building via national and international partnerships. Should the national partnerships element of the programme be significantly expanded this would alter the balance in a way which may be prejudicial to the overall delivery of the programme.	June 2024 (completed) September 2024 March 2025
Co-ordination and collaboration with other actors:
6	CCG should continue to review opportunities for collaboration with the Africa Energy Futures Initiative, as and when it starts work. If there is an overlap of interest in terms of country programmes there may well be opportunities to find synergy in efforts to strengthen national research capabilities in CCG’s areas of interest. Response: Partially accepted	Priority: medium	Partnership Director
CCG Management Response/Strategic Action Plan	Timeline
The Climate Compatible Growth (CCG) programme will engage with the Africa Energy Futures Initiative (AEFI), proposed to support home-grown African analysis, once it is fully established. CCG also recognises that its engagement must transcend this initiative, and aim to effectively influence the global energy landscape by expanding its reach and actively engaging with international initiatives through its tools and analysis. For instance, participating in umbrella initiatives like the Global Energy Alliance for People and Planet (GEAPP) is crucial, as these platforms provide opportunities to collaborate on a global scale and navigate complex international energy and climate dynamics. In line with this broader engagement strategy, CCG has already made some important strides. 6.1 Discussions with AEFI have been initiated to collaborate on knowledge products and tools to help unlock critical climate finance and drive investment in clean energy solutions. 6.2 Work has begun to feed off each other; for example, the Starter Data Kits (SDKs) feature in AEFI latest Science paper. 6.3 CCG has taken proactive steps to build institutional capacity by engaging in joint initiatives with leading African academic centres, such as Strathmore University in Kenya and the University of Cape Town in South Africa. These efforts are part of a broader strategy to ensure that African institutions are at the forefront of sustainable energy research and development. Furthermore, CCG is contributing intellectually to the formation of the African Institute for Sustainable Energy and Systems Analysis (AISESA), a key institution that could play an important role in helping to address Africa's institutional and human resource challenge. Through its Engagement Fund, CCG has also provided material support for AISESA’s upcoming in-person meeting in Cape Town, scheduled for mid-September 2024, which will further solidify these collaborative efforts.	ongoing
Management:
7	FCDO should continue to do all it can to provide greater certainty around annual budget allocations in advance of any financial year. Response: FCDO to determine	medium	Programme Management
CCG Management Response/Strategic Action Plan	Timeline
7.1 The annual budget for 2024−25 was shared through the Letter of Variation (reflecting FCDO Accountable Grant Agreement (AGA) budget revisions). 7.2 A projected CCG consortium budget distribution for 2025−2030 was presented at the FCDO Workshop on 14 June 2024. 7.3 Furthermore, CCG budget adjustments are undertaken and confirmed on an annual basis, to allow adjustment for arising priorities, as and where applicable. 7.4 The clause in the FCDO AGA, for treasury approval beyond March 2025, has currently added some risk/uncertainty with regards financial contractual commitments beyond this period. 7.5 In parallel, outside CCG influence, pressure on the government to revert the FCDO review cycle to a three-year period is encouraged, which will support greater programme certainty. CCG will continue regular dialogue with FCDO to ensure that this is achieved.	March 2025 subject to Treasury approval of CCG budget
8	FCDO should provide guidance on how far CCG may deviate from the original intention in terms of the proportion of funding allocated to the different elements of the project (programmed research, partnerships, the SPF and so on). See also recommendation 3. Response: FCDO to determine	low	FCDO
CCG Management Response/Strategic Action Plan	Timeline
8.1 CCG is fully receptive to FCDO’s guidance on this recommendation. 8.2 CCG’s quarterly/annual reports to FCDO and/or quarterly workshops to highlight early warning should issues be identified.
9	Once the current recruitment of two new senior staff is completed CCG and FCDO should carefully monitor and assess whether those changes to the size and structure of the existing senior management team and the administrative staff are sufficient to ensure both strategic and administrative management of the programme or whether further staff resources are required. Response: Accepted	high	Programme Management
CCG Management Response/Strategic Action Plan	Timeline
9.1 CCG has completed the recruitment of the two senior staff, Romana Rauf as Head of Programme Portfolio joined in April and Jairo Quiros-Tortos will join in August as Deputy Director. Furthermore, to ensure that effective current and future management of the CCG flexible funds (FRF, Engagement, Agile etc.) and reporting processes, the programme recruited an additional Project Manager and is in the process of recruiting an administrative support role for this part of the programme. 9.2 CCG has recognised that due to the expansion in the volume of International Partnerships Activities and to enable to support future effective delivery of this part of the programme, additional central support would be required. To address this, CCG in process of recruiting a full-time member of staff to manage EMP capacity-building events and the community of practice. The programme has already appointed an administrator supporting overall international parentship activities. 9.3 CCG and its partner consortium have further identified additional technical and administrative, research associate, and junior staff positions for further recruitments. However, given the break clause in the FCDO AGA requiring treasury approval for budget beyond March 2025, a few casual short-term positions are being filled on a temporary basis instead. 9.4 The CCG Programme Management team is in the process of re-structuring and re-allocating roles and responsibilities within the team to prevent longer term overstretching of staff members. The team will continue to actively monitor current and new programme requirements and workloads of individual staff members, and in this way it will ensure that potential requirements for additional resources are identified in timely manner. However, this can be challenging in instances where funding for short-term projects with imminent start times is awarded to CCG, as staff recruitment and onboarding processes require time.	March 2025
10	CCG should implement its recently finalised VFM framework, which should be viewed not just as a guide for an annual reporting exercise but, as a management tool to help CCG ensure it both uses resources economically and allocates resources strategically. It should also consider refreshing its due diligence reviews of consortium partners as a matter of good practice. Response: Partially accepted (with justification) Based on lessons learnt in the implementation of the Value for Money (VfM) framework, CCG is in process of reviewing it to revise it further for a better, fit-for-purpose management tool, and after agreement with FCDO will implement the revised VfM framework.	medium	Programme Management
CCG Management Response/Strategic Action Plan	Timeline
10.1 CCG has agreed to provide FCDO with suggested revisions to the VfM Framework with supporting rationale by the end September 2024. 10.2 Building on the recommendations made by Oxford Policy Management (OPM), and the Annual Review recommendations, CCG has established a task group to review the VfM framework to create a better, fit-for-purpose approach, which includes (but is not limited to): 10.2.1 Revise external benchmarks to assess and ensure economic efficiency in various spending categories; 10.2.2 CCG found the allocation of accurate project costs against each projectisation deliverable to be challenging, and it is currently reviewing project budgets at appropriate macro levels, which will be further piloted into 2024 to inform the revised VfM framework; 10.2.3 CCG is furthermore reviewing the definitions and related benchmarks to develop more consistent mechanisms for reporting and quantifying co-funding and in-kind support. 10.2.3 Revise the practicality and scope of the existing mechanisms for measuring programme efficiency, effectiveness, cost effectiveness, and equity and related benchmarks. 10.2.4 Improve mechanism for integrating data collected on gender and equity considerations in programme outputs and engagements, to the extent that data are available, to promote inclusivity. This will be aligned to the ongoing revision of the ToC and LogFrame accordingly and will be further informed by the Gender Equality and Social Inclusion (GESI) strategy for CCG, which is currently under development.	31 October 2024
11	CCG should update its due diligence reviews for KTH, the Climate Parliament and the Centre for Global Equality. Response: Accepted	medium	Programme Management
CCG Management Response/Strategic Action Plan	Timeline
11.1 Climate Parliament due diligence has been received. 11. 2 CCG programme management is diligently following up with the Centre for Global Equality (CGE) and KTH Royal Institute of Technology. Both institutions have received the relevant documents and responses are anticipated in Sept (to factor the leave period).	October 2024
Research commissioning:
12	FCDO and CCG should consider whether, rather than budgets being based largely on incremental changes to a previous year’s allocation, it might be appropriate to use the annual consortium review process recommended in the 2023 Annual Review to more actively influence the allocation of funding across consortium partners, taking in to account both performance against research strategies and new opportunities that may have arisen in the preceding 12 months. Response: Partially accepted CCG is in agreement that a more strategic approach to budget setting is needed, but that will likely be considered for multi-year (possibly 3-yr) budget planning cycle within its Phase 2.	high	Directorate
CCG Management Response/Strategic Action Plan	Timeline
12.1 CCG has committed to drafting a Process Note for the Annual Consortium Review (ACR) and sending it to FCDO for review by the October 2024. CCG and the FCDO have negotiated the development of both an annual Consortium Review and performance diagnostic, as well as deep three-year strategy reboots for the Programmed Research Strategies. The latter allows Programmed Research Units to leverage PhD and employment cycles that allow delivery of higher value for money and deep research dives. Those are to be externally reviewed to ensure that they represent world-class frontier research. 12.2 With respect to the ACR, CCG management takes note of the FCDO email message dated 28 June which states that ‘the main purpose of the recommendation for an annual consortium review process (ACR), which is to enable a discussion, initially amongst CCG’s Directorate and then with the FCDO, on what different workstreams are delivering in relation to their budgets, and the reasons for the variations in this. FCDO understand that there will be significant variation. Some workstreams and projects will produce more outputs per pound, while others will take longer and require more resources to produce outputs, due for example to the rigorous methodologies used, onerous data gathering requirements, and a host of other valid reasons. Within the FCDO’s Research and Evidence Directorate we hold decades of experience of commissioning different types of research and have strong awareness of the different types of research outputs and the associated variations in their budgets. It is important that the ACR process does not create perverse incentives to produce more outputs for the same budget when those outputs may be of lower quality and / or less transformational for other reasons. It is a priority that CCG academics retain decision-making leadership on how CCG outputs are produced and delivered. In reviewing the ACR we will be fully open to discussing the reasons behind variations in outputs compared to budget across workstreams. We do not intend to use the ACR as a blunt tool to allocate budgets between workstreams. However, the outcomes of the ACR process may raise concerns, for example if there are cases in which a workstream’s contribution to CCG’s objectives are lower than anticipated or lower than others and there are no clear reasons for this. In these cases, the concerns raised will need to be discussed and approaches formulated to addressing them.’ 12.3 From CCG's perspective, focusing solely on outputs or immediate workstream contributions in the ACR can be misleading. This is partly due to the nature of research, which often involves long-term projects and indirect outcomes that are not fully captured by short-term outputs. Additionally, our demand-led and partnership-focused approach further emphasises the need for a more comprehensive assessment. Therefore, performance should be assessed/evaluated not only based on outputs but also on outcomes, research, and partnership strategies, adopting a strategic, holistic analysis of the programme’s overall delivery and mandate. 12.4 The more strategic approach to budget setting will likely be CCG’s multi-year (likely 3-yr) budget cycle and corresponding settlements within the Phase 2 that will allow for planning of academic style funding of posts and projects, which allows for a more strategic deeper review at its half-way pivotal point. 12.5 For annual monitoring of Consortium Performance against plans, the workstream projectisation tool continues to be the management monitoring tool which also informs on the links between consortium spend and activities. 12.6 The ACR and 3-yearly strategy reboot will form the basis of a more strategic approach to resource allocation which will be developed in coordination with the FCDO.	Dec 2024
13	CCG should consider if further effort is needed to incentivise a more strategic approach to collaboration across Programmed Research workstreams. Response: Accepted	medium	Directorate
CCG Management Response/Strategic Action Plan	Timeline
13.1 CCG is considering this as part of its new Programmed Research Strategy. The Programmed Research Leadership (PRL) is currently looking at a better way of structuring the workstreams to manage long-term strategic research and demand-led activities (as per recommendation 3) and to promote collaboration. Workstreams are not always conducive to effective collaboration. Therefore, CCG, informed by proposals from the PRL, is considering a new structure consisting of research communities and ‘solution initiatives’. The latter are cross-cutting themes that draw on the expertise of the research communities. This will – it is envisioned – foster greater integration within and across Programmed Research, as well as integration with Country and International Partnerships. Further, the proposed structure aims to foster more strategic resource allocation, balancing stability with agility and ensuring resources transparently reflect commitments. This new structure also aims to improve programme communication. In the past, a level of research integration has been achieved through sprints (short-term and time-bound efforts commonly originating from general meetings or external requests, such as the green hydrogen think-piece or critical minerals). The new Research Strategy is considering whether sprints should continue, and we recognise that they should not be the only mechanism for better integration and should be incorporated into the workstream structure. 13.2 The Research Directorate is in dialogue with Country Partnerships (CP), as country strategies can support the research workstreams to think about how collaborative CCG research activities may be embedded and self-sustaining beyond the CCG programme. This will help ensure balance between research that is strongly demand-led at a country level and that which is more driven by the wider academic field. This will help facilitate and guide the respective CP teams to focus their engagement and partnering with strategic stakeholders in order to generate outcomes and impact within respective countries. 13.3 The new revised SPF structure that will be drawn up will also be an opportunity to foster integration from the bottom-up through SPF projects that are co-created. This will be considered as part of recommendation 4.	Dec 2024
14	With respect to the operation of the FRF fund Several steps could be taken to improve the quality and operation of the FRF facility going forward: Guidance documentation could be reviewed and strengthened to improve clarity, particularly with regards to eligible costs. In future, CCG should consider whether the calls could be simplified by making budget limits consistent across all topics in an FRF call. Consideration should be given as to whether it is beneficial to commit to a certain number of funded projects per topic, or whether funding should be allocated to the highest quality projects across all topics. More consistent external reviews of projects should be sought. Individuals providing poor quality reviews should not be used in future, and the CCG team should consider and implement steps to guarantee that all proposals are adequately reviewed (this could involve extending the timetable for reviews, or increasing the number of reviewers to ensure enough good quality reviews). Response: Partially accepted CCG does not agree to make budget limits consistent across all topics in an FRF call because scope and associated costs for each research topic can vary substantially.	medium	Research Director
CCG Management Response/Strategic Action Plan	Timeline
14.1 Guidance documents have been reviewed and strengthened based on feedback provided. We have revised the application process to include an Expression of Interest (EOI) stage followed by invitation to submit a full application for shortlisted applicants, to improve the quality of proposals to be evaluated. The EOI stage will allow us to provide clearer guidance on (in)eligible costs for applicants progressing to the proposal stage. For FY5, a request for EOIs for nine research topics was launched on the Loughborough University procurement system on 15 July. We expect a Request for Proposals for shortlisted applicants to be released by mid-August. Proposals will not be marked down because of the inclusion of one or two ineligible costs; this can be addressed during contract negotiations. Final budgets for each FRF topic will be determined during contract negotiations. In the request for EOI, we have provided a broad budget range as guidance; however, the full submission budget in phase two of the process will vary across FRF topics dependant on scale and associated costs. Applicants will be given at least three weeks to prepare proposals. 14.2 The EOI process will also help us to determine appropriate budget ceilings for the various FRF projects and the number of projects funded per call. We feel that this is a better approach to determine budgets per project rather than having similar budgets across all projects as per the recommendations, so as not to restrict the flexibility of the scale of project deliverables, country context, and how the funds can best be used. Furthermore, in the review process, it is possible that not all topics attract at least one high-quality proposal. 14.3 We have improved the review matrix and are providing more structured guidance to improve consistency of the reviews. There will also be a pre-screening through the EOI process. External reviews of proposals will be used where necessary to help inform final decisions on awards but not all submitted proposals will go to external review as the initial evaluation can be done by CCG in-house to save time and reduce costs without impacting quality. We will maintain an updated list of reviewers who perform the task well for future FRF calls. 14.4 CCG has recruited an additional Project Manager (and is in the process of recruiting administrative support staff) to ensure effective management of the FRF finance and project monitoring. This will ensure that the management of this fund requires lower input of academic time. 14.5 CCG is working towards a mitigation mechanism for FRF underspend: 1) Through up to 10% overcommitment. 2) Building a reserve roster of fundable back-up projects, subject to availability of funding. Given previous years’ experience of underspend, the additional 10% of budget allocation will mitigate this concern. 14.6 CCG intends to build a roster of vetted external review experts; however, this will not be in place for the 2024 FRF review process. 14.7 The FRF process will be revisited for future calls. See recommendation 3.
Partnerships:
15	With respect to SIGs CCG should ensure that: SIG members are clear on the purpose and terms of reference of the SIG they are a member of and that they understand what membership demands of them. CCG should consider whether or not it makes strategic sense to support SIG members’ calls for more SIG meetings and / or more cross-SIG interactions. CCG should set out more clearly, where services it has inaugurated through its partnerships are intended to be sustained beyond the life of CCG, how this will be achieved. Response: Partially accepted We agree that greater clarity and focus should be articulated within the Special Interest Groups (SIG), and there should be a document; but it should be live and iterative, rather than being a fixed document (i.e., ToR)	low	Country Partnerships
CCG Management Response/Strategic Action Plan	Timeline
15.1 Special Interest Groups (SIGs) that are newly formed from FY5 will co-create a Statement of Purpose document to allow for open dialogue regarding the focus of the SIG. Then, within the SIG section of each country strategy, this Statement of Purpose will be input. The SIGs will also need to supply a draft future engagement plan to be placed into the country strategy. 15.2 The Country Partnerships (CP) team has decided that it makes strategic sense to encourage more SIG meetings and cross-SIG interactions, as such the team is providing financial support for SIGs for the first time in FY5. The SIGs are to align themselves to their respective CCG country’s strategy document and think through ways of contributing to the country strategy. Support started in April 2024 and will run until March 2025. The CP team will be evaluating how this support enhances (or not) national ownership, outputs from the SIGs, and network development. 15.3 Each CP team based within a Partner Country has developed a workplan for FY5 on how will support the SIGs, with activities and meet-ups scheduled throughout the year. These were completed in May 2024. 15.4 The latest iteration of the CP strategies includes a Monitoring and Evaluation table which provides targets and goals to be set by the end of the CCG programme (i.e., 2030). Within this table, there are indicators that highlight if the activity is required for ‘self-sustaining’ CCG activities. These indicators are aligned to the thinking around the Partner and Affiliate Countries Sustainability Strategy, which will be completed by March 2025. In brief, the Partner and Affiliate Countries Sustainability Strategy document will outline how the CCG programme plans to embed knowledge and expertise to ensure these are self-sustaining practices within the countries CCG engages in. This contrasts with what typically occurs in programmes, which tend to leave behind a vacuum – we aim to intentionally review what, ideally, the CCG programme will leave behind once it is finished in March 2030.	March 2025 May 2024 (completed) September 2024 and March 2025
Positioning for future impact:
16	In order to optimise its efforts to influence and push the boundaries of academic thinking CCG should, going forward, prioritise efforts on securing publication of peer reviewed journal articles that shed light on the key research questions identified in the workstream research strategies. Response: Accepted	medium	Directorate
CCG Management Response/Strategic Action Plan	Timeline
16.1 CCG Programmed Research publishes in a diverse set of journals, depending on the disciplinary field, nature of the method, and topic area. This is standard academic practice and is reflected in the publication records of the different PIs. Due to the time lag for journal publications, the evaluators would have gained a view of early publications based on the first projects completed. These would therefore not be fully reflective of the research strategies and questions, largely because strategies are sequentially implemented over the course of the five-year programme. In some cases, elements of the research strategies would not have been taken forward. Going forward, the research directorate will monitor the implementation of the new strategies to ensure that peer-reviewed publications are targeted as the primary output of research projects. 16.2 The diverse journals that CCG publications have appeared in (year 4) illustrates the range of research we are producing: Energy Strategy Reviews; Energies (MDPI); Global Policy; Environment Research Letters; Nature; Science Data; National Science Review; Data in Brief; Renewable and Sustainable Energy Reviews; Journal of Industrial Ecology; Energy Strategy Reviews; Nature Sustainability; Frontiers in Sustainability; Transport Policy; and Methods.
17	CCG should make efforts to broaden the selection of journals it seeks to have peer reviewed articles published in to maximise its reach to relevant academic audiences. Response: Accepted	medium	Directorate
CCG Management Response/Strategic Action Plan	Timeline
17.1 The target journal for research, as is common academic practice, is determined by the PI / author team. While journal selection is not something that can or should be determined by the Research Directorate, we recognise that some journals have been targeted more than others. The reasons for this include agreements with editors to run special issues, journals that provide a specific focus, for example publishing data (Data in Brief) or software (MethodX), journals that are open access, or journals that enable early career researchers to publish more easily (for example, those that are less competitive). In of itself, the journal selection is not a problem as long as the research being produced is of a high quality, supports the ToC, and helps meet LogFrame indicator targets. To ensure the latter, CCG has developed a quality assurance process, and where needed, CCG researchers can request editorial support. In addition, CCG will monitor the analytics regarding journal publications and provide these in the next annual report. 17.2 The Research Directorate will continue to encourage diversity of publication to ensure a wide reach and academic influence while also ensuring consortium and southern partners can publish in journals appropriate for their research and aim to progress to more prestigious journals as capacity is being built through strong partnerships and collaborations. 17.3 CCG has increased its effort to promote the dissemination of research. The CCG communications team is actively promoting research outputs across various channels and mediums. 17.4 CCG has developed a detailed quality assurance process for CCG outputs that also lists different journals commonly used in the space that CCG works within. It is, however, at the academic’s discretion to select a journal that is appropriate and aligns with the specific research.
The relevance of CCG’s capacity strengthening activities:
18	CCG needs to explore how its modelling training summer school alumnae could access further continuing support, post training, to help them apply their new skills in their work. Response: Accepted	low	International Partnerships
CCG Management Response/Strategic Action Plan	Timeline
18.1 This recommendation is very much in line with management thinking and consistent feedback received from EMP alumni for some time. In response CCG will launch a new Energy Modelling Community (EMC) in August 2024, an International Partnerships Activity (IPA) designed to provide ongoing demand-driven technical support to EMP alumni. The design of the EMC has been informed by a survey of participants at EMP-Africa 2024. All EMP alumni will be automatically enrolled in the EMC. The aim of the EMC is to ensure that skills gained from EMP participation are maintained and upgraded, and that members grow in confidence with the application of the tools in their daily work. 18.2 The EMC will offer alumni a range of support activities, including biweekly troubleshooting sessions on demand, as well as a monthly enrichment programme, which in any given month may comprise supplementary training sessions, clinics where members may present and request feedback on their own work in progress, and seminars showcasing interesting new model applications from within and beyond the network. The programme of activities will be co-designed with the membership via biannual townhalls. Alumni reunion and networking events will also be held annually at COPs. 18.3 Like the EMPs, EMC support activities will be delivered to sub-communities based on specific model interests. During a pilot period from September to December 2024, the EMC will launch with two technical tracks: one for OSeMOSYS and one for CLEW, as these are the two largest alumni sub-communities numbering 50−100 people in each. Beyond the modelling sub-communities, the EMC will also facilitate the formation of country level chapters, as well as cross-cutting interest groups, such as for female alumni or Francophone alumni, and potentially offer a mentoring scheme. 18.4 The formation of EMC will be complemented by the introduction of an Open University LinkedIn badging system to formally recognise different levels of attainment in modelling skills. Alumni of EMP will earn a Silver Badge equivalent, while the EMC will specifically focus on supporting alumni to learn higher level Gold and Platinum Badges, as well as Trainer badges to support alumni playing an increasing role in the delivery of capacity-building activities.	August 2024
19	CCG should articulate and implement a more strategic approach to building research and analytical capabilities in universities and research institutions that can then be drawn on by governments. This requires more careful thought about equitable partnerships and how much CCG should prioritise fuller engagement and participation of southern research institutions in core CCG research workstreams. There may be a trade-off between coherence of research agenda and long-term capabilities of southern partners to deliver ongoing research collaborations with their governments and other local partners after the CCG programme has finished. This recommendation should be viewed in conjunction with recommendation 5. Response: Accepted	high	Directorate
CCG Management Response/Strategic Action Plan	Timeline
19.1 CCG acknowledges the strategic approach and need for greater South–South collaboration. While much of CCG’s programmed research is researcher-driven (primarily from the North), CCG prioritises government request(s) to the extent possible, and generally links them with in-country academic partners, fostering collaboration that has the potential to last beyond CCG. It also builds the capacity of southern researchers to enable them to engage with government. At the same time, via the reformulated Southern Partner Fund, CCG will second southern researchers into CCG’s core innovation and research activities. This recommendation is accepted with the caveat that the expectations thereof are realistically aligned to the CCG annual resources made available for ongoing engagement and strengthening of southern research institutions into core CCG research workstreams. 19.2 CCG’s approach is to develop in-country self-sustaining knowledge ecosystems that allow for involvement of Southern Researchers within CCG’s Programmed Research. This is complemented by the Country Partnerships and International Partnerships parts of the programme, which create the conditions and incentives needed for productive collaboration among academics, government analysts, and decision-makers, fostering strong, practical connections. This is accomplished through combined capacity-building and co-creation activities, community support, and the embedding of research within countries via special interest groups and university programmes – each with tailored incentives to encourage collaboration. Additionally, CCG supports southern researchers in securing new external funding and offers incentives and political economy training to empower analysts, guiding both academic researchers and government technocrats toward sustained engagement beyond the life of the CCG project. These are elaborated below. 19.3 The CCG programme actively fosters collaboration between academics and government technocrats, analysts and policymakers. This collaboration fuels research with practical applications, ultimately informing government policies. CCG ensures all its outputs are open access (following a set of principles know as the U4RIA principles) and readily available to government analysts. 19.4 CCG is developing three primary sets of activities and a coordinating function to address the International Partnerships (IP), Country Partnerships (CP), and Programmed Research (PR) workstreams and their role in coordinated capacity development. At the time of evaluation, these efforts were in early stages, but significant progress has been made. This includes a special focus on increasing capacity, agency sustainability, and inducing research outputs that are policy relevant from southern partners. The initiative was kicked off in the Summer General meeting of 2024 and is currently spearheaded by the Partnerships Director with the Programme Director. 19.5 A core objective of the programme is to elevate the role of southern partner researchers for engagement with government counterparts. This includes but is not limited to: 19.5.1 CCG capacity-building and co-creation Energy Modelling Platforms (EMPs) are designed to promote the science-policy interface in several ways. The EMP’s provide high demand (and competitively awarded services) to hundreds of government technocrats and academics in deep partnership with a set of International Organisations. First, participants are not only equipped with technical skills, but also prepared to better understand the political context of their work by taking the mandatory prerequisite training in "Political Economy Analysis" (PEA). Second, the Energy Modelling Platforms (EMPs) are deliberately designed to mix participants from universities and government agencies, fostering collaboration and relationship building throughout the process. Third, the research projects that participants pursue during these capacity-building events are always grounded towards addressing live policy questions in their own countries. Moreover, at the end of EMPs, analysts are trained to present and pitch their modelling results in a way that is accessible to policy audiences These efforts are already yielding tangible results. In Uganda, an EMP alumnus played a crucial role in advising the government on the development of a national energy transition plan. This success led the IEA to implement a "bootcamp" approach, sending government officials to future EMPs to enhance their capacity for co-creating the national energy transition plan – which was subsequently launched at COP28. Building on this success, Uganda, in collaboration with the French Development Agency (AFD) and CCG, has initiated a Data-to-Deal (D2D) project. And, Uganda has now joined the growing number of CCG Affiliate Countries, which are countries in CCG's target regions of Africa and developing Asia that partner with CCG on a targeted set of activities. D2D specifically focuses on large-scale finance mobilisation, the primary goal of CCG as described in CCG’s impact statement. Concurrently, the Ugandan academic lead is actively implementing the "Flatpack" MSc programme at their university, integrating CCG tools into the curriculum. 19.5.3 At the country level, CCG works with partner universities to establish "special interest groups" (SIGs) that typically include academics and government representatives. These groups collaboratively define research priorities and ensure its practical application. There are variety of ways CCG works to include government officials into the SIG, but it does depend on country context. For example, in Ghana they have required that the co-lead of a SIG be a government representative. However, in Lao PDR, it is different, and government members may act as unofficial advisors for the SIG group rather than be delegated to be directly on the SIG. Alternatively, it may be that a SIG comprises predominately government officials, such as the energy systems modelling SIG in Lao PDR. We should note that most SPF, FRF, and Engagement Fund projects are encouraged to have government members on the project, meaning they are still involved in SIG activities. It should be noted that the experience of the CP team is that including government officials to be a part of the SIGs has been manageable, and nearly every single SIG across all CP countries has at least one government representative. 19.5.4 With respect to Programmed Research, as Phase 2 commences CCG aims to build upon successful examples of collaboration between government and academia (eg Lao PDR energy modelling and green hydrogen initiative), and embed the underpinning principles from these examples in future collaboration. Core in this is the inclusion of research organisations in co-creation activities from the outset, establishing groups that include CCG researchers, government stakeholders, and local academia in delivery of policy-facing outcomes. This is designed to work alongside efforts by International Partnerships to facilitate uptake of CCG’s ecosystem of methods, tools, and data (eg EMP, Catalyst Facility) which can be coordinated with the deeper and targeted Programmed Research engagement, and to leverage Country Partnerships SIGs on the topics of those deep engagements. 19.5.5 CCG is considering developing an output indicator for its monitoring framework, specifically focused on the co-development of knowledge products with both government and academic partners. This further emphasises the importance of collaboration in building a robust knowledge base for informed policymaking. 19.5.6 CCG will provide the FCDO with a strategy for long-term engagement of Global South researchers with CCG work streams by Jan/Feb 2025. Furthermore, by March 2025 CCG will develop a strategy setting out how it will ensure long-term sustainability of partnerships in Partner Countries and Affiliate Countries. To that end, contributions from southern partners are categorised as follows: i) “absorbing” input from southern partners, ii) “co-created” with southern partners, or iii) led or “induced” by southern partners. This classification was discussed with the Operational Intelligence Framework (OIF), and an indicator has been introduced to track the number of research outputs led by southern academics. Various activities in IP, CP, and PR are structured to foster this southern-led “induced” research, and build the capacity needed to execute it. The CCG Engagement Fund, currently in a prototype phase, is designed to help realise outcomes. An example of an outcome includes the embedding of research capacity in national universities, promoting long-term sustainability. As induced research expands, CCG's Monitoring, Evaluation, and Learning (MEL) framework will explore how to better integrate these outputs into the programme’s strategic vision. 19.5.7 International Partnership (IP) products (examples of which are briefly described below) are designed to support large-scale analytical and research activities that national governments can use. This approach, as outlined in Cannone et al., Addressing Challenges in Long-term Strategic Energy Planning in LMICS: Learning Pathways in an Energy Planning Ecosystem (2023), demonstrates early success with southern partners contributing to policy, publications, and academic programmes. These IP products are deployed in Partner Countries via the CP activities and in Affiliate Countries through direct CCG engagement or via international partners. While integrating peer review into these efforts remains a challenge, steps are being taken to address this. The key IP products include: (i) Energy Modelling Community (EMC): A community of practice supporting long-term, southern-led research. It comprises technocrats and academics that are alumni of the Energy Modelling Platform (EMP) schools. They receive regular communications from CCG and have meetings, including regular seminars. (ii) Energy Modelling Platforms (EMPs): Two-week (or more) capacity-building events for technocrats and academics. Admission to the EMP requires a letter of commitment from their seniors (dean or director-general) confirming that their EMP work will contribute to government-supported research. Participants must also complete the OpenLearn Collection (OLC) capacity-building course for their respective track and work on a policy or research-relevant case study. In addition, they produce reports and datasets (co-created with CCG academics), with some of these progressing to peer-reviewed papers with support from the Knowledge to Paper (see further below) initiative. Others contribute to policy documents. (iii) IP Flatpack: A programme integrating OLC teaching materials into MSc curricula in southern universities. (iv) OpenLearn Collection (OLC): Online courses providing students with certification badges. All lectures and notes are freely available for download to southern researchers under open-source licences. OLC courses point towards IP's Data and Tools offering. (v) Data and Tools: An IP business line offering free and open-source data and tools via online platforms. (vi) Knowledge to Paper (K2P): This nascent initiative, in its second iteration, provides coaching to EMC members to translate their reports into publishable papers. CCG is supporting the publication of a special issue of a peer-reviewed journal (initially, Energy Strategy Reviews) focusing on southern-led research using CCG knowledge products. We call this type of research ‘induced’, as – rather than co-created – it is led by southern researchers. All the research is policy facing. (vii) The recently established Catalyst Facility is a trust fund designed to strategically build research and analytical capabilities within southern universities and research institutions – in a manner that incentivises the link with government policy analysis. This aligns with the CCG objective to strengthen the capacity of governments to draw upon robust research for informed policymaking. The Catalyst Facility addresses the ask by: Prioritising equitable partnerships: By requiring government endorsement and engagement in all Catalyst Facility activities, it ensures that research efforts are aligned with national priorities and that southern researchers are actively involved in policy-relevant research. Fostering long-term research collaborations: The MSc Scholarship Window encourages students to intern with government institutions, fostering connections between academia and government. The Research Collaboration Window explicitly requires academics to develop proposals for external funding, promoting sustainable research collaborations beyond the CCG programme. All windows emphasise the importance of government engagement and support, laying the foundation for ongoing research partnerships. Building research capacity: The Academic Training Window enhances the skills and knowledge of southern academics in energy modelling and research methodologies. The Sabbatical Window provides opportunities for advanced research, publication, and capacity building at renowned research institutions. Key features that demonstrate a strategic approach: Government endorsement: All applications require a letter of support from a government counterpart, ensuring alignment with national priorities and facilitating government engagement in research activities. Focus on policy relevance: The Catalyst Facility emphasises the importance of research that directly informs policy decision-making. Sustainability: The programme encourages the development of sustainable funding sources and fosters long-term research collaborations beyond the CCG programme. By implementing the Catalyst Facility, CCG develops a proactive approach to building research and analytical capabilities in the South, ensuring equitable partnerships and fostering sustainable research collaborations that will benefit governments and academics and improve the science–policy interface long after the CCG programme concludes. 19.5.8 Further, based on country demand (and drive), CP adopts an “Inclusive Innovation Approach”, which builds productive interdisciplinary, inter-sectoral, and international collaborations to help create an enabling environment for national actors. CP, as of FY5, has offered its institutional partners a Network Activities budget to support SIG and network activities, which include research, capacity building, and participation in key events (eg Kenya’s Clean Cooking Week). To date, these funds have enabled more induced research which is solely led and driven by members within the Partner Country networks (and its SIGs). This additional budget to the CP teams in the partner countries also fosters a strategic approach in thinking about how these funds could have a deeper impact within the country and aligns with Country Strategy as well as the CCG LogFrame. 19.5.9 PR activities include efforts to co-create and induce research. A notable example is the publication by Kihara et al. (2024: Mid- to long-term capacity planning for a reliable power system in Kenya), led by the Kenyan Ministry of Energy, with support from CCG. This work exemplifies how CCG can facilitate both academic agency and integration with government decision-making. Additionally, the Southern Partner Fund (the content of which is overseen by PR), is currently under process review to ensure that it is embedded into the core programme, following budgetary challenges that previously limited its scope. Successful embedding is anticipated to come with deeper Southern Partner capacity, agency, and exposure. As PR strategies are being ‘rebooted’ there are active coordination efforts with IP and CP, as well as alignment with aforementioned updates to the CCG LogFrame which encourages engagement that induces research and embeds capacity in national universities. (Due to budget cuts and uncertainties, SPF deployment was made on an internal competitive, project-by-project basis, which often employed southern researchers involvement in CCG core research, but mitigated against long-term capacity-building engagement of southern researchers in the core programme.) 19.5.10 CCG tracks co-funding as a metric of success. The majority of co-funding is developed in a tripartite relationship with local academics, government, and CCG researchers. In a partner country setting, an example includes the PACT Kenya project. This is an effort of more than a million GBP which supports the development of the Integrated National Energy Planning process and includes Strathmore and various government organs. In affiliate country settings, examples include efforts in Sierra Leone which enables university and Ministry of Energy engagement. This is partly because there is a large cohort of EMC members in government and universities, so this often allows for quick entry and access to counterparts that (if not already connected) are easy to connect. In the case of Ghana, EMC members in the Kwame Nkrumah University of Science and Technology (KNUST) and the Energy Commission have enabled CCG to work collectively to set up components of an energy planning academy. This draws from a Flatpack activity being developed at KNUST.	March 2025
20	CCG needs to decide if the ‘teaching kit’ website remains a priority and, if so, ensure sufficient resources are allocated to completing the transfer of materials to it. Response: Accepted	low	International Partnerships/KTH
CCG Management Response/Strategic Action Plan	Timeline
20.1 Early feedback from universities participating in the Flatpack Programme has indicated that they prefer to make use of existing learning materials available from OpenLearn Collection (OLC) and EMP, rather than the teaching kits due to limitations of the software interface. CCG management has therefore put a hold on any further conversion of learning materials into teaching kits and is pursuing the Flatpack Programme with existing learning materials that are proving to be fit-for-purpose. Once the Flatpack roll-out is established, two options for the curation of source material will be explored. The software used to enable this curation will be accessible to CCG as well as to partners. This is to allow partners to develop their own courses, potentially drawing from CCG course material, or (subject to criteria and review) to add to CCG OLC material directly. One option is the further development of the teaching kit software as originally designed. The other option is the use of the OLC platform’s “Moodle” software directly. Both will be explored as options. User experience and curation needs will be among the criteria used to inform the exploration.
Early evidence of outputs and intermediate outcomes:
21	CCG should develop a more robust approach to testing evidence supporting claims of progress at the outcome level. This could include triangulation (seeking confirmatory evidence of a change from more than one source) and an effort to understand and document the multiple factors that may have influenced that change. Response : Accepted	medium	OIF
CCG Management Response/Strategic Action Plan	Timeline
21.1 CCG will revise the Outcome Reporting methodology by the next General Meeting to make the current approach more robust. This may include one meeting with the FCDO prior to the final presentation of a final methodology. 21.2 CCG is currently engaged in reviewing the LogFrame and ToC. Ongoing discussions include potential adjustments in several key areas: Major Changes under consideration: Enhancing GESI Reporting; Incorporating subnational/local governments to ‘OI.1 CCG knowledge products used for decision-making’ and 'OI.2 Nationally embedded processes; Modifying the Definition of Indicator ‘OA 1.4 Capacity Building’; Modifying ‘OI.1 CCG knowledge products used for decision-making’ to better capture support from LMIC institutions; and Addressing the gap in capturing network building as an outcome. Minor Changes under consideration: LogFrame Streamlining – Incorporating ‘OA 2.4 Review Articles’ and ‘OA 3.1 Sprints’ into other indicators; Making ‘OA 3.2 Policy and Programming Engagement Across HMG’ non-scorable; Adding ‘Acts of Parliament’ to the language of ‘OI.1 CCG knowledge products used for decision-making’; and Including ‘FRF Kick-off workshops’ to the language of ‘OA.1.3 Uptake and engagement activities’. Changes to Theory of Change under consideration: Strengthening Integration of GESI; Better assessing and addressing Political Economy and systemic developments (CCG Political Economy Unit); and Improving the balance between quantitative and qualitative research. 21.3 Once these changes are approved by the CCG Directorate and agreed to by the FCDO, they will be implemented in 2025. While OIF will lead the reviewing of targets, the quality assurance and implementation thereof will rest with the responsibility of workstream leads/consortium partners, with OIF providing an oversight audit role. This process is aimed to be finalised by the end of October 2024.	End Oct 2024
Gender and social inclusion:
22	CCG should finalise and implement its draft GESI strategy and, in the process, confirm a mechanism for meeting the commitment to track the extent to which female and LMIC researchers lead research or are lead authors in publications. Response: Accepted	medium	Directorate/RD
CCG Management Response/Strategic Action Plan	Timeline
22.1 CCG has implemented a more rigorous way of capturing disaggregated data for authors in output reporting which now includes reporting on female and LMIC researchers leading research or who are lead authors in publications. 22.2 The team leading the CCG Gender Equality and Social Inclusion (GESI) efforts is currently exploring the appropriate mechanisms to monitor and capture female and LMIC research leaderships, as well as to promote this more actively. 22.3 The CCG GESI strategy has been drafted with a clear plan for implementation. It will be launched in January 2025. 22.4 A joint workshop with United Nations Office for Project Services (UNOPS) is planned for the first quarter of 2025, and it will bring together infrastructure planners and investors to explore GESI within the context of energy and transport infrastructure, building on CCG work and in line with the CCG GESI strategy.	October 2024
23	CCG should continue to monitor the level of female attendees in its modelling training to confirm if its recent commitment to a minimum proportion of female trainees on modelling courses has the intended effect of raising female participation. Response: Accepted	medium	International Partnerships
CCG Management Response/Strategic Action Plan	Timeline
23.1 CCG will continue to monitor closely participation of female students in its capacity-building services. Statistics already available from the EMP indicate that the acceptance rate of female applicants is similar to male applicants, but that fewer women apply to the capacity-building programmes. CCG will partner with female associations such as UNESCO’s Women in Science and the Global Women’s Network for the Energy Transition to promote more female applicants to EMP, which is currently the main barrier limiting female student numbers. To illustrate a representative way of measuring gender representation at CCG capacity-building events reference is made to the example of CCG Summer Schools (EMP events). Since 2021, the total percentage of female EMP event participants has been increasing (25.19% in 2021, 36.14% in 2022, 28.76% in 2023 and 29.17% so far in 2024). Please note that in 2022 CCG held two events, one of which was a small-scale online EMP-LAC event with 50% female participants, hence a higher than usual increase for that particular year). 23.2 In addition, CCG has introduced a mechanism to monitor the gender of students completing OLC certification, which is not currently known, by requiring online participants to complete a registration form. 23.3 CCG will introduce, within the new Energy Modelling Community (EMC), events and activities for specific support of female membership. In the second semester of 2025, EMC will organise a webinar for female alumni to exchange experiences regarding the challenges that women face in the energy planning field, and to elicit feedback from female alumni regarding what kind of support could be most relevant. As part of EMC’s planned mentoring programme, a special female mentoring programme is under consideration.	December 2024
Inclusive growth
24	In implementing the new draft GESI strategy, once finalised, CCG should monitor its impact on how well inclusive growth is reflected as a focus of CCG research. Response: Accepted	high	GESI lead
CCG Management Response/Strategic Action Plan	Timeline
24.1 The CCG GESI strategy is being finalised with support from UNOPS. Since the evaluation and with input from UNOPS, the implementation of the GESI strategy has progressed, especially regarding different aspects of CCG such as country engagement and different groups like disabled individuals and refugees. 24.2 Task forces for the various priority areas are being created to ensure dedicated effort through consultation before early 2025. 24.3 A baseline for each of these priority thematic areas is being established, which will help monitor progress over time and adjust the set expectations. 24.3 CCG acknowledges the finding that its work on the distributional aspect of climate-compatible growth could be stronger, given that ‘equitable growth’ is part of the CCG ToC’s impact statement. Nevertheless, the GESI strategy alone cannot be the vehicle for responding to this. Furthermore, limitations to the availability of GESI data have been factored into the current review to enhance GESI reporting revisions to its LogFrame/ToC. 24.4 The development of distributional analysis in impact assessment in CCG’s Data-to-Deal approach is being developed in partnership with CCG’s economics, investment pipelines, and International Partnerships research activities. Further, deep analysis is being undertaken in Partner Countries Zambia and Kenya to enable analysis to include GESI in study design, especially as it applies to distributed energy deployment into regions where many of the most vulnerable are located. 24.5 Further consideration of how (or whether) CCG will deepen its consideration of the distributional aspects of growth will take place in discussion with the FCDO once the GESI Strategy is finalised. Which aspects of the GESI strategy that are implementable are dependent on the budget that is made available through additional external funding.	Baseline established by March 2025 Ongoing

Table 2 examines selected key findings that are mentioned in the Evaluation Report that may not have reached the level of a recommendation, but that nonetheless we wish to acknowledge and comment on.

TABLE 2 – Other Key Findings (Phase 1 External Evaluation Report)

Other Key Findings (Phase 1)	CCG Management Response/Strategic Actions (taken post Phase 1)
Section 3. Findings on Programmed Research: 3.3–9.2 Workstream performance via comparison of number of outputs/journal papers. 3.5.2 Workstream Activities and Outputs “Table 10 summarises the outputs to date across Workstream 4 from all three universities, who have all been working on different activities. The cumulative funding allocated to this workstream to date (including the budget allocation for FY 23/24) is approximately £1.64 million, making it the second largest programme research workstream in financial terms. Whilst it is difficult to make direct comparisons between the different outputs of different workstreams, it is noticeable that with a budget equivalent to eighty percent of Workstream 3 has produced less than half the number of outputs of that workstream and, from an output perspective is at the same level as workstream 5 (which has only around sixty percent of workstream 4’s budget).” (p.21).	The new Research Strategy, currently under development (to be finalised before the year-end) includes consideration of workstream performance towards the CCG programme goals, and is open to various options, including review and restructuring of workstreams for a more conducive approach to integration (see the response to recommendation 13). With regard to this point, it is also useful to cross-reference recommendation 12 in the core evaluation, which sets out the need to move to a more certain 3-year cycle of Research Strategy updates, to ensure that research units have time to develop and deliver more transformational research publications. Moreover, the variation in outputs between workstreams (per pound spent) is a challenging feature of CCG research. This is because some research effort takes longer to produce outputs than other research efforts (e.g. because the research aim is ambitious, substantial data gathering is needed, methodology development needs revision to ensure high quality results, to name a few). It is important to emphasise that counting articles published is likely to be a poor way to incentivise research more generally, or to judge research quality, because it encourages rapid publication of lower-quality material in journals with arguably lower peer review standards, which we would like to avoid. CCG management do accept the need to understand and communicate research activity, so in response to these points in the evaluation we propose to put in place a reporting mechanism to track the progress of low or modest workstream outputs (WS 4, 5a, 5c referenced in the evaluation). In practice, this will be achieved through the use of the new project management software (ClickUp) once the programme projectisation is fully transitioned (anticipated by summer 2025), where we will support workstreams to identify outputs that they are working towards and have them report on progress towards producing them. The Oxford Programme for Sustainable Infrastructure Systems (OPSIS) group in Oxford University has continued to explore demand for infrastructure resilience modelling in Country Partnerships. CCG participated in the annual workshop in Accra in September 2024, where CCG was able to present CCG research to a range of stakeholders, as well as holding bilateral meetings with government officials. Strong demand for climate resilience analysis is reflected in an FRF project being proposed on climate resilience of the electricity transmission network. Officials from the Department for Transport expressed interest in transport resilience analysis, and further interest was expressed by the Ghana Infrastructure Investment Fund. We are following up on this interest and will explore potential responses.
Section 5. Findings on national partnerships 5.2 Strategy [Data-to-Deal: Transport Application – Figure 6] “Given that the Data-to-Deal framework seems to be an important core concept for CCG, more work may need to be done to ensure it is consistently communicated and applied across the programme” (p. 56).	With regards to the evaluation discussions on CCG transport research, the reference was made to a figure on D2D in transport that was developed during a very early stage of the D2D development. This figure was used for internal purposes to provide an overview of the different activities (including FRF, SP, IPAs and Programmed Research activities) and how they relate to each other. Therefore, some of the initial key elements of the D2D approach (data collection, modelling, finance) were borrowed to create a specific CCG transport-related narrative. The figure was never published nor used publicly and was never envisaged as a formal D2D application. However, in the meantime, CCG has begun developing a transport-specific D2D application. In partnership with the Asian Development Bank (ADB), a workshop on D2D in transport had been held during the ADB Transport Forum in Manila in May. ADB is very interested in the D2D approach and CCG recently finalised a common report for the ADB (post evaluation phase 1) with a focus on transport modelling. The collaboration continues and there are plans to present the first policy brief on D2D in transport at COP29 and to publish a detailed report in February 2025.
Section 5. Findings on national partnerships: 5.4. Kenya country case study: 5.4.7 Observations “CCG could consider updating the wording of outcome 1 of its Kenya strategy so that it is very clear that it aligns with its remit and sphere of control” (p. 63).	CCG has accordingly revised the wording of Country Partnerships outcomes and intermediate outcomes in the Kenya country strategy and that of other countries. Programmed Research (PR) activities are also being aligned with this strategy, with examples such as the Kenyan Ministry of Energy’s leadership in co-created research.
Section 6. Findings on modelling capacity strengthening efforts: 6.5 Observations on capacity strengthening efforts: 6.5.2 Early results and effectiveness “training will only really be effective in levering institutional change in planning approaches if it is delivered in concentrated form […] and if follow-up support is available” (p.77)	CCG’s International Partnerships team is working on improved targeting and delivery of capacity building and associated impact through ‘bootcamps’. Bootcamps are a sustained package of capacity-building services provided to a group of academics or policymakers in a specific country that have the specific goal of delivering a piece of academic research or a policy document (such as a plan or strategy). Once a request for a bootcamp is received, CCG will allocate a dedicated coach to ascertain capacity-building needs, providing training via the Energy Modelling Platform, The coach will accompany the team through the EMP training to ensure that different skills acquired are being effectively applied to the production of the intended document and continue to provide support for up to 6–12 months thereafter. The response to recommendation 18 regarding the newly formed Energy Modelling Community is also of great relevance to this finding.
Section 6. Findings on modelling capacity strengthening efforts: 6.5 Observations on capacity strengthening efforts 6.5.3. Efficiency “[…] the open learn materials are clearly valued […] assuming this activity is to continue for the remainder of the CCG project, then budget and management effort needs to be allocated to completing the transfer of material to the ‘teaching kit’ website and also to setting out a clear strategy […]” (p. 78).	CCG intends to develop the teaching kit website into a more complete repository of teaching materials relevant to the CCG modelling ecosystem, including lecture materials, exercises, and quizzes. The website will act as a curated repository for materials developed by CCG and other partners, and it will provide open access materials to instructors globally.
Section 8. Findings on management: 8.2. Resource allocation Section 8.2.2 Impacts of constraints on size of Southern Partners’ Fund (SPF) “Overall, Southern research institutions are not very involved in ‘pure academic’ research, which is primarily led by CCG’s university consortium. There are, however, a few instances where Southern researchers have been involved in publishing academic research. An example would be the recent paper submitted to Energy Strategy Reviews on how to link OSeMOSYS to FlexTool, based on the experience in Kenya, with the lead author (Mugai Kihara) from the Ministry of Energy. There was also significant involvement of Southern authors in the development of policy briefs commissioned for COP26 and COP27 (for example at least 12 of the 21 CCG COP27 briefing papers included authors from LMIC institutions), although this practice has not carried over to COP28.” (p. 90)	CCG supports the delivery of research by southern researchers not only by directly funding them to produce research (as set out in the following paragraph) but also through a range of cross cutting complementary activities. For example, CCG is advancing new initiatives to build southern researcher capacity and foster long-term partnerships. This includes classifying southern contributions as input-based, co-created, or induced, with a focus on increasing the number of research outputs led by southern academics. Key International Partnership Activities initiatives like the Flatpack Programme, Catalyst Facility, and Energy Modelling Community (EMC) are designed to embed research capacity in national universities and support southern-led research. Furthermore, The Flatpack Programme integrates CCG's transition modelling tools into MSc curricula in the Global South, ensuring local academics can effectively deliver these courses. The Catalyst Facility provides fellowships to southern researchers, while the (EMC) is a community of practice that is supported by CCG to induce the production of policy-relevant research. These efforts ensure southern researchers lead policy-relevant projects and develop peer-reviewed publications that address national challenges, thereby ensuring long-term sustainability and impact. To date, the Flexible Research Funds have by-and-large been awarded to southern research institutions (via open call), which often carry out academic research. For example, in the 2023 call, 73% of the funding was awarded to southern researchers. Furthermore, as partnerships evolve, often built from Southern Partnership Fund projects, more southern researchers are being brought on to work alongside CCG researchers to produce impactful academic research relevant to their countries. For example, the paper Leonard et al. (2023: Shedding Light on Vulnerability: Mapping Intersectional Needs for Inclusive Energy and Development Planning) included a Kenyan academic. However, one should note the structural differences between academic institutions in high-income countries (HICs), which often have a strong focus on advancing academic knowledge, and institutions in low- and middle-income countries (LMICs), whose primary mandate is teaching. Therefore, any involvement in such research activities often takes place outside of their working hours. With new resources provided to the Country Partnerships team to deliver the country strategies, along with the Engagement Fund, FY5 has shown an uptake of academic leadership from southern institutions in CCG Partner Countries.
Section 8. Findings on management: 8.5 Programme Management 8.5.1: Communicating direction, decisions and progress internally Lack of clarity on budget allocations Section 8.5.1 “Some areas where further thought may be needed were identified during the evaluation though. Although expressed in different manners, they could largely be grouped around the need for more senior management feedback and direction in what is a very complex programme. This was sometimes voiced in terms of the challenges of understanding and managing budgets in a complex environment: There is a challenge as a team lead with CCG in this respect as you don’t have information on the financial allocations made to the different members of your team in different universities, so even if you wanted to manage budgets you have no information to do so. There is an agreement at the start of the year with the researchers in the team on what is expected in terms of outputs / outcomes for the year and the XX team meets every couple of weeks to discuss progress, but that is the limit of the ability to manage resources within the existing structure. Things are further complicated by the nature of CCG, with contributions around XX for example made across different WS’s and IPAs, which stitch together to create outputs and outcomes. (KI93) There is a challenge around clarity of budget, as it is not clear what budget is available to conduct a capacity building event. There is more clarity around co-funding than internal CCG funding (KI24)” (p.96)	CCG has increased its management staffing with the recruitment of the Deputy Director, Head of Programme Portfolio, and Project Manager, and it is in the process of recruiting additional staff as CCG expands its operations in preparation for subsequent phase. This has increased engagement and quality of communication with the Directorate, the PIs, and team leads and partners. There are ongoing efforts for timely sharing of information for cross-cutting CCG strategic pathways. In addition to annual general meetings, the Directorate holds weekly meetings. Furthermore, workstream PIs engage in fortnightly meetings and Workstream teams engage in monthly projectisation meetings, which has led to more efficient information sharing. Overall budget allocations between Programmed Research, Management, and International and Country Partnerships are set in agreement with the FCDO and are documented in its Business Case and the Business Case Addendum. This guiding structure provides oversight in managing CCG's complex programme, while also allowing flexibility for responsive action, as needed. Programmed Research will submit a three-year Research Strategy to the FCDO, and as long as it delivers to plan, it is allowed to operate without deviation. The three-year cycle allows Programmed Research to deliver deep research without being pulled to compelling demands; at the same time it enables stronger value-for-money as it aligns with three-year PhD cycles around which resource and momentum can be crowded-in (not possible with annual contracting). Similarly, budgets for specific activities, such as capacity-building schools, are developed within the envelope agreed upon for International Partnerships. Progress is evaluated annually through a Consortium Review process. Programmed Research also has access to additional budget for responsive activities through its Agile Fund, and more recently, Country Partnerships has access to Engagement Funds (implemented after the evaluation Phase 1 completion), which are agreed upon in collaboration with the FCDO to address emerging needs. Continued efforts are ongoing to provide timely clarity of the budget allocations, which will be part of the annual Consortium Review and within the timeframe constraints of FCDO budget cycles. This includes ongoing efforts to improve communicating the direction of decision-making and progress internally within the Consortium.
Section 8. Findings on management: 8.5 Programme Management 8.5.1: Communicating direction, decisions and progress internally “There was also feedback concerning the level of communication and support from senior management both to provide feedback and guidance for workstreams as they manage their own work and to provide an overarching view of CCG as a whole, progress against strategic goals and how important decisions are made: There is some confusion around the hierarchy and areas of responsibility/seniority within the CCG structure. CCG is large enough that more formal structure is required. Lack of clear structure can make it difficult to prioritise when there are conflicting demands and reduced efficiency of delivery. (KI24)” (p.96).	CCG acknowledges the requirement for establishing areas of responsibility/seniority within the CCG structure and hierarchy. CCG has not only recruited more management resources (after Phase 1 evaluation), but also set up a formal structure (Q1 of 2024) to better communicate roles and responsibilities as well as formalised the representative voting membership within the Directorate. Furthermore, CCG has initiated a culture of cross-cutting representation in working groups, for example working groups to develop/update Value-for-Money, Monitoring and Evaluation, Enhanced Risk Management Strategy frameworks amongst others. The CCG project manual is also in process of being updated for communication of the many processes for ease of reference of the otherwise complex CCG structure. The recently appointed Research Director is also engaging PIs for regular meetings and for an inclusive approach to develop the new Research Strategy.
Section 8. Findings on Management: 8.6 Overall observations on management: 8.6.1 Efficiency “In addition, expenditure has heavily favoured modelling as opposed to other forms of research and energy as opposed to transport. FCDO and CCG management need to decide if any of these imbalances need addressing or whether the current balance of resource commitment reflects priorities for the remainder of the CCG project.. Likely as a result of the complexity of the CCG structure further effort is needed to ensure that individual workstreams are provided feedback on their work, that their efforts are managed to ensure the sum of CCG’s individual parts is optimised, and that there is a good understanding across all the consortium partners of the progress CCG is making against its strategic goals and the basis of the central decisions that are made to keep it on track.” (pp. 97–98).	CCG has developed a revised “country strategy” programme to provide future direction and vision of our programme in respective countries, the implementation of which remains to unfold. This interfaces with the establishment of a Political Economy Unit designed to better understand country-specific demands and identify areas where CCG can have the most significant impact. By aligning these demands with CCG’s Programmed Research portfolio, the programme aims to enable better convergence and create transparency with respect to the demand and supply of CCG knowledge product co-creation. This effort is part of a broader process that includes consortium reviews and periodic Programmed Research reboots every three years to support the health and direction of CCG’s Programmed Research. Additionally, the programme is undergoing a comprehensive review of its research operations, portfolio structure, and International and Country Partnerships interfaces, with an emphasis on enhancing collaboration across workstreams. The issue of insufficient collaboration between workstreams, such as those focused on modelling, is being addressed through a significant Programmed Research Strategy reboot in 2024. This reboot seeks to develop a new integration proposal and strategy by the end of the year, which will sharpen alignment and synergy across all workstreams. A follow up reboot is planned for 2027, with annual updates to those reboots in-between. In terms of International Partnerships Activities (IPAs), of which there were several, CCG is streamlining its approach by consolidating these activities into three distinct units, each guided by a clear strategy and operational plan.