Department of Performance Monitoring and Evaluation (DPME) Annual Report 2012/13

Meeting report

Department of Performance Monitoring and Evaluation (DPME) Annual Report 2012/13
Dr Sean Phillips, Director General (DG), Department of Performance Monitoring and Evaluation, said that the focus areas of the Department were the following programmes: monitoring and evaluation (M&E) of national priorities, GWM&E system, management performance monitoring and evaluation, and frontline service delivery.

The Auditor-General South Africa (AGSA) had awarded the DPME a clean audit with no material findings on financial statements or on compliance with laws and regulations and no material findings on performance information. Regarding material adjustments to performance information, the Department was implementing a system of centrally filing all evidence documents and auditing of quarterly and annual performance reports by internal audit had been introduced. Several changes had been made to information technology (IT controls): the Department had migrated to its own information communication technology (ICT) system at the end of March 2013; user access policies had been updated; revised controls were implemented; and it had implemented its own policies and disaster recovery system.

The DPME’s listed its key achievements for 2012/13. The Department had supplied Cabinet with quarterly monitoring reports on implementation of the 12 outcomes and provided the public with quarterly progress reports on the website. Research had begun on the 20 year review and 23 research papers had been produced. A national evaluation plan was developed which was approved by Cabinet in December 2012. 24 evaluations (from 2012/13 and 13/14 evaluation plans) were under way by the end of the financial year. In addition, management performance MPAT assessments on 156 national and provincial Departments was conducted; a Municipal Assessment Tool was developed and was being piloted; the resolution rate of the Presidential Hotline was improved from 82.2% to 90.2% for national and from 49.8% to 71% for provincial departments; over 215 front-line service delivery monitoring visits were conducted; and a national citizen-based monitoring policy framework was developed.

Summary of Performance per Programme
Programme one was administration. The number of targets was 32, 24 were achieved (81%), 4 were partially achieved (13%), and only 2 had not been achieved (6%). The final Strategic Plan was tabled to Parliament by the due date, the Annual Performance Plan (APP) was tabled to Parliament by the due date and all quarterly reports were submitted, the communication plan was approved by the Director General and quarterly progress reports were produced, and the risk register was approved. The 3-year rolling internal audit plan was approved by the Audit Committee, quarterly internal audit reports were compiled and submitted to the Audit Committee and management,  all remaining recommended (but not mandatory) polices as per regulatory frameworks and collective agreements were approved, all policies were reviewed, but no changes were required. The Department achieved a clean audit opinion, and the Annual MPAT self-assessment was completed by management and signed off by Director General by 30 September 2012. Enterprise Architecture was approved (including Information Strategy, IT governance framework was approved, IT Procurement Policy was approved, data management policy was approved, record management policy and file plan was approved, internet and email policy were approved, hardware policy was approved, asset management policy was approved, architectural design was approved by the Director General, and DPME successfully migrated to its own IT system by the end of March 2013.

The only targets not achieved were that the security plan was in draft form and not approved and the
production of development indicators had been deferred to the current financial year.

In programme two, outcome monitoring and evaluation, there were 14 targets. Of these, 13 were achieved (93%) and one was partially achieved (7%). The Department did not fail to achieve any targets. The summary outcomes monitoring reports on four quarters were compiled and submitted to Cabinet, 233 briefing notes on Cabinet memos were compiled, 115 briefing notes or reports on executive monitoring and evaluation initiatives were compiled,  the annual plan for 2013 was approved by Cabinet and the three-year plan for 2013-16 was approved by Cabinet,12 guidelines to support evaluations across government were produced and published, evaluation and competency standards were produced and put on the website, 237 government officials completed training on evaluation 12 courses, governance structures for the 20-year review were put in place and 22 research papers were received by the end of the financial year. 10 evaluation reports were approved by evaluation steering committees, one evaluation report was approved, two more reports were approved by the evaluation steering committees in May 2013.

The evaluation process had been proven to be more complex than anticipated by the targets. The process of working with departments and obtaining agreement on the various stages of the evaluations had proven to take longer than anticipated.

Programme three, M&E systems coordination, had 14 targets. 11 were achieved (79%) and only 3 were partially achieved (21%). The Department did not fail to achieve any targets. A baseline of M&E systems was established from the first MPAT assessment report, a diagnostic assessment of human resource capacity for M&E across government was conducted and improvement plans were developed, the concept and project plan for M&E system assessment across government was developed and implemented. The Preliminary Integrated Capacity Development Programme was implemented, an M&E learning networks event was held, 28 data forums were held, 52 new datasets were fully assessed, data sources with acceptable metadata description increased by 18% and quarterly reports were presented to the data forum and top management within the time frames. The Draft Results Bill was submitted to Cabinet for approval by March 2013, the draft Bill was developed but consultations with other administrative centres of government departments did not result in agreement. There were different views on the need for a Result Bill amongst the key administrative centre of government departments. The revised GWM&E framework was submitted to Cabinet for approval by March 2013, a draft GWM&E framework was completed, but was not ready for submission to Cabinet. The document required extensive development through a series of iterations as well as wide consultation with key stakeholders. That process had now been combined with the process to develop a Results Bill through developing one overarching discussion document, which was currently in the cluster system.

Programme four, M&E Public Service Oversight (PSO), had 18 targets. 17 were achieved (94%), one was partially achieved (6%), and no targets had not been achieved (0%). The MPAT1.2 was released on 28 and 29 August 2012 to all national departments and the Premiers’ Offices, all departments completed their MPAT assessments before the end of the financial year. 21 national Director Generals and 79 provincial Heads of Department provided electronic sign-off on their assessments. The MPAT result report was compiled and submitted to Cabinet. 65% (101 out of 156) of departments showed an improvement in at least one of the standards in each of the four key performance areas on the comparison between the MPAT 1.1 and 1.2 self-assessments, 100% (15 out of 15) of the departments which submitted their APPs on time were assessed and letters were sent to the 15 departments which submitted their 2nd draft APP to DPME by the due date, and seven Forum of South African Directors-General FOSAD reports were prepared and presented. Improvements were achieved for 12 indicators being monitored by FOSAD. Frontline Service Delivery Monitoring (FSDM) implementation tool and guidelines was published on the DPME website, national visit scheduled was finalized, and 215 site visits were conducted, the National Overview Report was produced and submitted to Cabinet by the end of the financial year, consultations were done with eight departments on the eight sector findings reports and annual findings reports were compiled, the return monitoring on 27 facilities was conducted and reports were produced on the 27 monitored sites for improvements as per the agreed schedule. 65% of service delivery sites which were visited at least twice had an improvement in scores for a least two of the seven assessment areas. Six case resolution reports for the Presidential Hotline were submitted to FOSAD Manco, the improvement plan was approved and four progress reports against the improvement plan were produced. The quarterly reports were tabled to top management meetings on progress against the approved technical support plan. 30 call centre agents were in place, and the case resolution rate was at 90.20% by 31 March 2013.

The Department achieved or partially achieved the vast majority of their targets. The only focus areas where targets were not achieved was administration, which had not completed its security plan or the
production of development indicators. In public service oversight, outcome monitoring and evaluation and in systems coordination the Department did not fail to achieve any of its targets. In total, 63 targets were achieved.

Human Resources
Mr Philips began the human resources report by discussing the vacancy rate. At the end of the 2012-13 financial year the Department had 197 funded posts of which 173 were filled, the vacancy rate at the end of the reporting period was 12%. The reasons for the non-achievement of the 10% vacancy rate target included: too much time was taken to complete compulsory pre-employment screening (two to three months delay in some cases), there was delay in Department of Public Service and Administration (DPSA) directive on implementing Public Service Co-ordinating Bargaining Council (PSCBC) resolution one of 2012 related to salary levels 9-10 and 11-12 , and in certain high level skilled positions, difficulties were experienced in recruiting appropriately skilled officials due to competiveness in the labour market. In addition, internal promotions as well staff leaving the Department had an impact on the vacancy rate (11 internal promotions and 17 exits).

During the reporting period the Department successfully filled two Deputy Director General posts, the Chief Information Officer (CIO) position had been vacant but had recently been successfully filled. In the 12 month period from 1 April 2012 to 31 March 2013, the Department had 17 exits. Turn-over rate was 12% (17 out of 143), due to officials leaving for promotions or to pursue other career opportunities. 104 posts in the Department were evaluated. The high number was mainly due to the implementation of PSCBC Resolution 1 of 2012 (grading of jobs on salary level 9-10 and 11-12).

The Employment Equity statistics at the end of the 2012-13 financial year were: people with disability stood at 2% (4 out of 173), African people stood at 79% (136 out of 173), and women at Senior Management Services (SMS) level 44% (16 out of 36). The Department was focusing on recruiting more females and Africans in senior management positions to ensure set targets were achieved as well as increasing the number of people with disability in the Department, more specifically in SMS. There were no disciplinary cases. There were three grievance cases received and resolved within the specified time frames. 137 employees and 147 interns received the necessary training. The PMDS policy was implemented in the Department and assessments for the 2011-12 were finalised in the second quarter of the financial year. 43% (59 out of 137) of eligible staff received a performance bonus. The Employee Health and Wellness Unit was established and a number of health programmes were rolled out.

Financial Information
The appropriation statement table showing economic classification, total budget per programme, as well as shifts and virements was summarised. There had been budget shifts and virements because all virements were done within the 8% limit for virements in terms of section 43(2) of the PFMA. In addition, there had been savings on compensation due to: difficulties in finding suitable candidates for some specialist positions, leading to recruitment processes taking longer than planned; a number of advertised positions were filled by successful internal candidates, resulting in new vacancies opening up; there was a delay in implementation of the Bargaining Council resolution regarding salary levels 9/10 and 11/12, which was budgeted for; there were some delays in obtaining pre-employment security screening results, which delayed some of the appointments; and there was a delay in implementation of Head of Department assessment programme.

Under programme one, savings on compensation moved to ICT unit for additional software licenses. Under programme two, savings on compensation moved to goods and services to fund the 20-year review and FOSAD research project on concurrent norms and standards. Some savings from Presidential Hotline (SITA and call costs lower than expected) moved to goods and services under OME branch, to fund the 20 year review and research on norms and standards for concurrent functions.

The graph showing expenditure overview per programme was summarised, as well the graph showing monthly expenditure such as salaries and wages, good and services, capital assets, and original projection. Overall expenditure of the graph was outlined indicating that R6 189 (3.6%) was under expenditure, R7 800 (4.5%) was committed but delayed, and R160 170 was the expenditure (see attached document).

The Department had spent 92% of its overall budget by the end of the 2012/13 financial year. There was an increase in expenditure towards the end of the financial year largely due to expenditure on IT equipment, evaluations and research projects. The Department was required to engage with South African State Information Technology Agency (SITA) to plan its IT infrastructure requirements. Those engagements were only finalised in June 2012 and the procurement of IT equipment could only start after that. That explains why IT expenditure largely took place towards the end of the financial year. The evaluation programme was still relatively new. The National Evaluation Policy Framework was approved in November 2011 and the 2012/13, national evaluation plan was only approved in June 2012.  That explains why expenditure on evaluations largely took place at the end of the financial year. The 2013/14 evaluation plan was approved in November 2012 and preparations for implementation started at the beginning of 2013. Therefore, expenditure on evaluations would be more evenly spread over the 2013/14 financial year.

The DPME was requested to carry out a 20 year review and norms and standards research project after the beginning of the financial year, and expenditure on those two projects had therefore largely taken place towards the end of the financial year. Reasons for under-spending of R13.9 million (8% of budget) were that funds were committed but expenditure was delayed, there was a R7.8 million (4.5%) delay in delivery of ICT equipment and finalisation of software development, there was R2.7 million (1.6%) in under-expenditure on compensation of employees (reasons provided earlier), and R3.4 million (1.9%) savings on Hotline expenditure (State Information Technology Agency and call costs were lower than expected).

Regarding goods and services, the hotline was transferred to DPME from the Presidency on 1 October 2011. For 2012/13 budget, DPME requested additional funding for Hotline for additional call centre capacity, increased services from State Information Technology Agency (SITA) and for calls made to toll-free number. Additional call centre capacity had been put in place during 2012, and additional services were being provided by SITA. Request for additional funding for 2012/13 was based on estimate from SITA and estimate of cost of toll-free number (at time of transfer, telecoms operators had not yet invoiced for toll-free number) and on actual SITA costs for increased call centre operators and increased services, and actual toll-free number costs that had turned out lower than estimated (resulting in saving of R2.9 million against budget).

As for capital expenditure, the DPME ICT infrastructure plan was agreed with SITA in June 2012. Procurement of hardware only started in July 2012. Migration to ICT system was completed by 31 March 2013. Migration to own telephone system was completed by end of June 2013. R2.7m was unspent on ICT projects due to R1.9 million in orders placed and delays in the delivery of equipment.

In concluding the presentation on expenditure, Mr Philips said that DPME was a relatively new department. Approvals for evaluation programme were only obtained after the start of financial year. DPME had now settled into multi-year budgeting and expenditure with planning and preparations taking place the year before expenditure takes place. It was now possible to start evaluation and research projects early in the financial year, which would ensure a more even spread of expenditure. Corrective measures were introduced to address under-expenditure. For the budget for the 2013/14 financial year, the Chief Financial Officer had played a more direct role in engaging with programme managers to ensure that budgets were realistic. The Department had also introduced monthly budget and expenditure meetings to ensure more accurate budgeting and more rigorous expenditure monitoring in future. Therefore from 2013/14 DPME expenditure was more evenly spread over the financial year. In addition, the Department had received R950.000 in aid assistance from GIZ and R944.000 was spent on two projects related to M&E capacity building. The Department did not incur any irregular expenditure in the 2012/13 financial year apart from expenditure on finance leases, automatically condoned through Treasury practice note 5 of 2006/07. Fruitless and wasteful expenditure amounting to R65.000 was incurred on travel-related cancellations and no-shows. R37.000 was condoned because it occurred as a result of unforeseeable and unavoidable circumstances and R4.000 was recovered from officials. The remainder was still under investigation at the time of the report but has since been either condoned or recovered.

2013/14 Quarter One and Quarter Two Expenditure
The Department had abolished programme three of M&E Systems coordination and support on its organisational structure and allocated its functions and staff to other programmes, in order to increase organisational efficiency and effectiveness. However, for the 2013/14 financial year, the budget for programme three remained. Data support moved to programme two, outcome monitoring and evaluation. Geographic Information Systems moved to programme one, administration. Budget for M&E Policy and Capacity building would move to programme four, Institutional Performance Monitoring, from 1 April 2014. The request for R8.4m in roll-overs was not approved. The remaining R3.7m for 20-year review and R4m for norms and standards projects was to be funded by reallocating funding from ICT, and Citizen Based Monitoring and Evaluations. The R19m in available donors funding would be used to ensure all APP targets were still met.
The graph showing actual expenditure and original projection was summarised. Expenditure planning for goods and services was on track with 39% of budget spent and a further 33% already committed (contracted). Procurement processes for ICT equipment for disaster recovery centre were in progress and would be contracted by October 2013. Acquisition process for additional office space was still in progress with Department of Public Works. The DPME would save money if this acquisition process delayed would be reallocated to evaluations if necessary. At the end of September 2013, 192 posts were filled out of 206 funded posts (93% filled). The Department was on track with respect to the recruitment processes.

In concluding, monthly expenditure monitoring meetings had significantly improved expenditure planning and forecasting, although there was still room for further improvement. The Department was on track to fully spend its budget by end of March 2014 and to avoid the large March spike which was experienced last financial year. Measures had been put in place to reallocate any possible savings to priority spending areas such as Evaluations and Municipal Performance Assessment.

2013/14 Adjustments Budget
Mr Philips summarised the table showing AENE reprioritization for 2013/14, indicating economic classifications, total budgets per programme, shifts and virements.  It was reported that reasons for AENE reprioritisation of funds included the following savings under programme four and the use of fund on new posts under programme two. There were other shifts and virements due to movement of functions. During the 2012/13 financial year, DPME was asked by FOSAD to fund research on norms and standards for concurrent functions. The total project cost of R5.2m of which R1m was spent in 2012/13. The roll-over was not approved by the National Treasury. There was R6.2m total cost of 20-Year review project. Roll-over of savings due to project delays in 2012/13 was not approved. Reprioritisation from programme three and programme four was done to fund norms and standards research and 20-year review projects under programme two. Some back-office ICT projects would be delayed to 2014/15 financial year to fund norms and standards and 20-year review projects.

2014/15 - 2016/17 MTEF Budget
During the 2011 MTEF cycle the Department requested additional funding of R449m per annum to increase capacity and to establish regional offices to enable the Department to fully deliver on its mandate. However, National Treasury indicated that our additional funding request could not be met. During the 2012 MTEF cycle the Department reduced its MTEF projections by 1%, 2% and3% respectively in line with instructions from Cabinet for all Departments to reduce expenditure. Going forward, DPME baseline increased by 4.8%, 4.3% and 6.0% respectively over the MTEF in nominal terms. The baseline therefore decreased in real terms given a projected CPI of around 5.5% and average cost of living increases of between 6% and 7%.

MTEF priorities included funding of new or expanding priorities within a budget allocation that decreased in real terms required significant re-prioritisation of funding within the Department. The following strategic decisions guided the process, namely; priority should be given to increasing capacity in evaluations and to extend management performance and service delivery assessments to local government , provision should be made to rent additional office accommodation for the Department, internal audit capacity would be increased from 2014/15, the outcomes support should be expanded to include two new outcomes from 2014, a Citizen Based Monitoring programme should be established and rolled out. The Department had followed a zero-based budgeting approach. Each sub-programme had provided budget estimates based on planned APP targets over the MTEF. Targets and budgets were adjusted in line with the priorities mentioned above. The table showing budget per focused areas was outlined indicating the total budget and % of each focused areas of M&E Policy / Capacity Building, MPAT, evaluation and research, administration, local government, and outcomes support. The table showing budget per economic classification was summarised indicating the total budget and % of each economic classification- capital expenditure, goods and services, and compensation (see the document for more details).

The presentation had showed what the Department could achieve, partially achieve and not achieve within current re-prioritised baseline allocation, translation of MTSF into new performance agreements and delivery agreements from 2014. The DPME could monitor progress against delivery agreement, could carry out evaluations of major and strategic government programmes, could develop and pilot Municipal Assessment Tool (25 municipalities), and assess municipal management performance of all municipalities. The DPME could monitor the performance of individual national and provincial government departments (MPAT),monitor frontline service delivery, develop Citizen Based Monitoring programme and roll-out to all 9 provinces, manage the Presidential Hotline (handle current call levels) and promote good M&E practices in government.

Discussion
The Chairperson noted that the results of MPAT were the same as the Auditor-General. He wanted clarity on the monitoring the performance on individual, national and provincial department. He asked who gave DPME clean audit.

Mr M Swart (DA) noted that the report was good and there was progress. He then wanted clarity no increase in budget for the DPME, and on funds from donors.

Mr N Signh (IFP) wanted clarity on the effectiveness on internal audit, increase in outsourcing and contracting, increase in travel costs, staff retention and the money owed to DPME by other Departments.

Mr G Snell (ANC) asked why the National Treasury denied DPME roll-over.

Mr Philips replied that donors generally preferred to fund bigger projects that would benefits the public. It was difficult for DPME to fund projects that involve management change. The work of DPME was attractive to donors as long as the DPME perform well and deliver. Internal control was managed but IT controls and internal auditing. The DPME agreed with Auditor General South Africa’s recommendations as to how the DPME should improve. Increase in outsourcing and contracts were largely due to increase in monitoring and evaluation work. The DPME had realised that it was not going to be economically if fulltime staff were hired, and that’s why the DPME had decided to outsource and contract. As time went on, the DPME would hire people with good qualifications and experience.  The Committee should note that monitoring and evaluation were two different things. Monitoring was more about tracking the performance of the departments and projects, and evaluation was more about in-depth detail study of a project. The traveling cost had increased due to too many trips domestically doing monitoring and evaluation work. The DPME’s staff turnaround was 10% which was normal in the public service sector.

The chairperson thanked the Director General for the presentation

The meeting was adjourned