2021-22 Statement of performance

Commission recommendations explored, agreed, and implemented:

•    How many inquiry recommendations were agreed and implemented?

•    How fully were the recommendations implemented or actively explored by the relevant policy makers?

Last year the Immigration inquiry made a series of recommendations to Government, no formal response has been received.

Understanding of productivity- related matters increases

Surveys, focus groups and expert review undertaken for the Immigration inquiry.

Expert peer review undertaken on the E+R team (Biannual).

Understanding of productivity-related matters

Survey results

Most survey respondents said the inquiry had increased their understanding “a little” or “a lot”. Responses to specific statements from the 74 respondents were as follows:

•    Immigration has played an important part in New Zealand’s economic development: 45.8% a little; 35.1% a lot.

•    On average, immigration is not driving down wages or replacing local workers: 37.8% a little; 35.1% a lot.

•    The immigration system currently uses a range of tools that may supress wages, job creation, and productivity: 41.9% a little; 32.4% a lot.

•    The Government should use an Immigration Government Policy Statement to improve the quality and transparency of immigration policy: 36.5% a little; 52.7% a lot.

•    The Government should engage with Māori in good faith on how to reflect Te Tiriti o Waitangi in immigration policy and institutions: 33.8% a little; 45.9% a lot.

•    The Government should improve the prospects of local workers instead of restricting immigration to prevent potential job displacement: 41.9% a little; 43.2% a lot.

Expert reviewer (Richard Bedford) noted:

In its ‘Briefing for the Incoming Minister (BIM) of Immigration’ in June 2022, MBIE refers to the Productivity Commission’s inquiry on pg. 4, noting that “the Productivity Commission has several findings and recommendations for the immigration system. You will receive advice on a government response in August”.

It will be interesting to see how many of the Commission’s recommendations relating to immigration policy tools, including preparation of an immigration Government Policy Statement, are adopted by the immigration policy teams within MBIE’s Labour, Science and Enterprise Group. (pg. 4 of expert review)

Focus group

All participants wanted the Commission’s final report to have an impact on policies and behaviours, however, no-one felt fully confident that would happen:

“I haven’t seen any changes based on the Commission’s report so far, but hopefully that will happen.”

One person commented that while the rebalance had made some changes to allow migrant workers some ability to change employers, there were still a lot of barriers to doing that.

People were looking for more acknowledgement of, and response to the inquiry from both the Government and MBIE. Decisions on sector workforce agreements and the Green List had cast doubt over how much influence the inquiry would have.

On a more positive note, it was acknowledged that work on immigration is ongoing and that there will of course be further policy changes and working groups to address particular issues in the future. Participants and their organisations planned to make good use of the Commission’s inquiry, especially the data, in those ongoing processes.

Understanding of productivity-related matters

Expert reviewer (Professor Robert Buckle) noted:

No formal evaluation processes have been carried out as no inquiries were completed during this reporting period, due to the timing of the inquiry process. With each inquiry taking 18–24 months, work carried out during this year will be evaluated in the 2022–23 Annual Report.

Third party commentary and reporting in the media

Survey

Of the 84 members of the public responding to the survey, they largely agreed that the Commission had engaged with the appropriate people and had consulted the correct information during the inquiry.

•    Sourced all relevant research and information: 47.6% (40) agreed; 17.9% (15) strongly agreed; 22.6% (19) didn’t know.

•    Engaged with the right people: 40.2% (33) agreed; 12.2% (10) strongly agreed; 31.7% (26) didn’t know.

•    Engaged effectively and appropriately with Māori organisations and individuals: 21.7% (18) agreed; 9.6% (8) strongly agreed; 62.7% (52) didn’t know.

Expert reviewer (Richard Bedford) noted:

Extensive media coverage followed the public release of the Commission’s findings and recommendations on 31 May 2022. There were over 110 references to these in the print media, on radio and television, and in online outlets between 31 May and 1 September.

This high level of interest in the findings of the inquiry is hardly surprising given the attention in the media to immigration generally at this time with the progressive opening of the border and the implementation of the Accredited Employer Work Visa system.

There has not yet been an official response from the Minister of Immigration to the inquiry’s findings and recommendations, but it is clear from the media coverage that these have generated considerable discussion and debate about immigration.

Focus group

Participants thought the Commission’s final report needed and deserved wide distribution, with some saying immigration tended to be seen more negatively than positively, and the Commission’s report could help to change that. The finding in the Commission’s report that immigration did not, on average, drive wages down was seen as particularly helpful to address a widespread misperception.

However, some were worried the report would “get lost on the shelves and gather dust”, with the focus switching to the immigration rebalance of 11 May.

In general, people were disappointed by the amount of discussion and debate generated by the inquiry.

“I haven’t seen much debate after the initial launch.”

“It was always going to be limited due to the narrowness of the report.”

Third party commentary and reporting in the media

Expert reviewer (Professor Robert Buckle) noted:

Articles on productivity related topics were published to reach a broad audience.

The new Director has initiated a series of op-eds published by the New Zealand Herald and Stuff. These articles focus on issues pertinent to the role of the Commission and signal the type of work undertaken by the Commission. This series has included articles on the benefits of innovation, competition and innovation, and education, written in a style that introduces readers to the role and work of the Commission.

They have a potentially important role in communicating to a broad audience why productivity matters for welfare and what can contribute to higher productivity. This seems an excellent initiative and an opportunity to extend the reach of engagement beyond the research and policy community.

Productivity analysis and advice improves:

•    % of inquiry participants surveyed who agreed or strongly agreed that the inquiry had helped to set or lift the standard in New Zealand for high-quality analysis and advice on [the topic]

6.3% strongly agreed

55.7% agreed

•    % of inquiry participants surveyed who agreed or strongly agreed that they will use the inquiry report as a resource and reference in the future.

24.4% strongly agreed

39.7% agreed

Productivity analysis and advice improvements

Survey

Most of the respondents either agreed or strongly agreed that the final inquiry report “focused on the issues most significant to the development of working-age immigration policy settings for NZ’s long-term prosperity and wellbeing”. 43.4% (36) agreed, and 21.7% (18) strongly agreed; 16.9% (14) disagreed; 10.8% (9) didn’t know; and 7.2% (6) strongly disagreed.

In a subsequent question set, the largest proportion of those who responded gave positive responses to the following statements:

•    The inquiry was sufficiently bold in reaching its findings and recommendations: 46.2% (36) agreed; 21.8% (17) disagreed; 14.1% (11) strongly agreed; 10.3% (8) strongly disagreed; 7.7% (6) didn’t know.

•    The inquiry has helped set or lift the standard in New Zealand for high quality analysis and advice on improving working age immigration policy settings for New Zealand’s long-term prosperity and wellbeing: 55.1% (43) agreed; 20.5% (16) disagreed, 11.5% didn’t know. The options for the terms strongly agreed or strongly disagreed both received 6.5% (5) each.

Expert reviewer commentary on the extent to which the research work: Helped set or lift the standard in New Zealand for high-quality analysis and advice on [the topic]

Expert reviewer (Richard Bedford) noted:

I am impressed by the extent and range of public consultation that the Commission undertook following receipt of the ToR for the immigration inquiry in April 2021.

With regard to the conduct of the engagement meetings, many of the early ones involved groups in round-table discussions with members of the inquiry team. The discussions were designed to inform the framing of the issues paper. Those academic colleagues and members of Ministries I contacted reported that these meetings were very open and constructive discussions. Genuine interest in getting ideas and feedback relating to the questions in the issues paper was apparent to participants in the panel discussions.

Focus group commentary on the extent to which the research work: Helped set or lift the standard in New Zealand for high-quality analysis and advice on [the topic]

Focus group

There was general agreement that the Commission’s inquiry had lifted the standard of analysis and advice on immigration policy:

“Yes, most definitely the inquiry lifts the standard.” “The report is of a higher standard than seen before.”

While one participant said that her organisation had always had good access to MBIE data, another was very appreciative of the data made available through the inquiry:

“It’s good to see data being available to everyone.”

Another perspective was that it was hard to know whether the inquiry lifted the standard.

“Prior analysis was carried out by MBIE behind closed doors, so it wasn’t possible to compare it with what the Commission had produced.”

Expert reviewer commentary on the extent to which the research work: Contributes to future work on [the topic] being better focused and use resource more effectively

Expert reviewer (Richard Bedford) noted:

To ensure that immigration policy is “fit for the future”, the Commission argues that a range of absorptive capacity issues must be addressed, along with how best to reflect Te Tiriti o Waitangi in immigration policy and institutions. To achieve this, the Commission recommends establishing an immigration Government Policy Statement (GPS) to improve the quality and transparency of immigration policy.

A major contribution of the Commission’s immigration inquiry has been to ‘red flag’ a number of key issues, such as absorptive capacity, that Ministries need to consider and address, rather than trying to do all the thinking for the Ministries.

In this sense, Immigration – Fit for the future and the other reports and working papers that have been produced during the course of the Immigration inquiry have made a particularly valuable contribution that extends well beyond the domain of immigration policy settings.

Focus group commentary on the extent to which the research work: Contributes to future work on [the topic] being better focused and use resource more effectively

Focus group

One participant suggested the Commission could usefully do a second immigration inquiry.

“This would look at the labour market impacts of recent immigration policy changes, and the opening of the border, over the coming two years. It could also analyse what happens to migrant workers six months and two years post-immigration."

A number of people, including those who had been somewhat disappointed with this inquiry, voiced strong support for the ongoing work of the Commission:

“Keep on consulting us and other stakeholders. We fully support the Commission’s work.”

Quality of analysis and advice in the inquiry and if inquiry reports will serve as a resource and reference in the future

Survey

In the survey, 39% of the survey respondents (30) agreed with the statement “I will use the inquiry reports as a resource and reference in the future”, with 24.7% (19) strongly agreeing.

Expert reviewer (Richard Bedford) noted:

A major contribution of the Commission’s immigration inquiry has been to ‘red flag’ a number of key issues, such as absorptive capacity, that Ministries need to consider and address, rather than trying to do all the thinking for the Ministries.

In this sense, Immigration – Fit for the future and the other reports and working papers that have been produced during the course of the immigration inquiry have made a particularly valuable contribution that extends well beyond the domain of immigration policy settings.

Focus group

All participants agreed that the Commission’s inquiry documents would be a reference and a resource in future:

“We will certainly use the report to help in understanding the labour market and when we make submissions on future immigration changes.”

Ongoing work that would likely benefit from the report and its findings included Regional Skills Leadership Groups, Workforce Development Councils, and the development of Industry Transformation Plans.

A number of those outside Government had already used the inquiry material in related work:

“We have already used the report to help put together a Select Committee submission and we will continue to use it.”

“Absolutely the research is becoming a source of direction for policy.”

Another participant said they had already taken actions because of the report, which had led directly to specific projects coming to fruition.

However, participants didn’t want the report to be just a reference. They wanted to see it used by MBIE and ministers to create policy.

Productivity analysis and advice improvements

Expert reviewer (Professor Robert Buckle) noted:

The Commission has increased the quality of engagement and collaboration with other stakeholders, particularly those in government policy agencies.

The reviewer commented on the re-activation of the Productivity Hub, with information about its activities now available online and open to Government and non-government researchers.

He also considered the initiative to “successfully resuscitate” engagement with the Government Economics Network and related research and policy groups within the public sector.

Relevance and materiality of inquiry report

Participant survey – Inquiry participants surveyed who agreed or strongly agreed that:

The Commission sourced all relevant research and information

47.1% agreed

18% strongly agreed

The Commission engaged with the right people

39.8% agreed

13.3% strongly agreed

The final report/ research paper(s) focused on the issues most significant to [the topic]

22.6% agreed

42.9% strongly agreed

The final report went into sufficient depth on the issues it covered

42.9% agreed

17.9% strongly agreed

Focus group

The Commission sourced all relevant research and information

The inquiry report, according to one group member, had provided medium and long-term solutions that contrasted with the Government’s ‘band-aid’ or short-term responses to immigration issues.

Another thought that the inquiry had reinforced an overdue need for our immigration settings to be simplified. This didn’t come so much from the inquiry itself, but from individuals describing the application process, visa criteria and so on:

“It seems to be atrociously complex.”

“If the inquiry achieves simplification that will be excellent.”

The Commission engaged with the right people

While most people agreed that the engagement was very good, some participants felt that the consultation process could have reached further and captured more voices:

“I felt the engagement was very good – I just wondered whether everyone had a chance to be involved.”

For example, one person mentioned that some of the smaller unions could have made useful contributions but didn’t appear to have been included.

Another mentioned that, ideally, the engagement would have involved a wider range of overseas business interests including fund managers, investors in NZ-based SMEs and listed companies, and the philanthropic sector. Potentially, these parties could have been brought together in one group to engage.

The final report/ research paper(s) focused on the issues most significant to [the topic]

There was a lot of support for the Commission’s final report, and the findings and recommendations it put forward.

“This was an essential investigation, and it was covered well.”

However, one participant commented that, in practice, it had a narrower scope that focused on work visas. The inquiry report would have been more useful had it delivered on the wider scope indicated in the Terms of Reference.

There was a second aspect in which the inquiry was seen as too narrow by one interviewee:

“I felt that the review itself took an almost exclusively economic reading.”

This person felt that the inquiry treated migrants as a pool of ‘short-term optional economic units’, used to plug gaps and make money for New Zealand businesses and the education sector. The many submissions with an international focus had reflected a much broader view of the importance and role of immigration, but they didn’t think this had come through into the final report.

“We [my organisation] felt that the Terms of Reference recognised a much wider range of factors that are critical for the two-way movement of people.”

The final report went into sufficient depth on the issues it covered

This question was not asked of the focus groups.

Relevance and materiality of paper(s) within the research work reviewed and the final inquiry reports

Expert reviewer (Richard Bedford) noted the extent to which:

• the Commission sourced all relevant research and information;
• the Commission engaged with the right people;

the paper(s) focused on the issues most significant to [the topic] and went into sufficient depth on the issues it covered.

The ToR covered a much wider range of topics and issues than had been suggested by the Commission.

The Commission is to be commended for developing such a well- designed and attractively presented invitation to a wide range of stakeholders, including the general public, to make submissions on aspects of the immigration inquiry between June–December 2021.

Just over 40% (15) of the 35 bullet points relate specifically to economic dimensions of migration, especially regarding the labour market (e.g., demand, wages, inclusion, ‘crowding out’), skills and education, business investment and incentives, and a range of adjustments relating to enhanced productivity, infrastructure provision (including housing) and macroeconomic phenomena such as interest and exchange rates and GDP growth.

Relevance and materiality of E&R work

Independent expert reviewer (Professor Buckle) noted:

With the support of the Board and additional financial support, the Commission had been able to “markedly improve the level of research experience, skill and capacity”.

However, he also provided feedback from interviews of economists and researchers on the most relevant issues:

“Some people interviewed thought the ERT could build on the excellent research and, as additional projects, identify how the insights could guide policy settings. Others argued that the ERT should concentrate on fundamental research on key issues associated with productivity and that it should be the responsibility of PC inquiries and relevant policy agencies to convert this research into appropriate guidance for public policy and management practice of firms.”

In an in-depth review of the report Export challenges and responses of New Zealand firms, Prof Buckle noted:

“This was collaborative work with New Zealand Trade and Enterprise and Text Ferret. The review noted the innovative approach of using a text mining approach, but that there was very little discussion of the selection bias, particularly explaining it for the wider audience for this paper including the policy community.”

The extent to which inquiry issues papers, draft reports and final reports were delivered to schedule

All external milestones communicated in the Commission’s process planning are achieved:

•     Inquiry processes

•     Research processes

All external milestones communicated in our planning in inquiry and research processes were achieved.

Milestones were monitored and reported in our triannual reporting to the Board and Ministers.

Participant satisfaction with the inquiry process

Inquiry participants surveyed who strongly agreed or agreed that overall, they were satisfied with the Commission’s inquiry process:

17.1% strongly agreed

65.8% agreed

Summary comments from focus group

All participants thought that the Commission’s process had worked well. When providing input to the inquiry, focus group members had been given the opportunity to comment at multiple stages. The Commission had provided them with both the information they needed, and enough time to participate effectively.

Summary comments from expert reviewer (Richard Bedford)

Between June 2021 and April 2022, the Commission prepared 10 research papers as part of the immigration inquiry. Most of these papers were subjected to external peer review and nine of them are available on the Commission’s website.

In addition, six substantive inquiries into specific issues linked with productivity and wellbeing dimensions of migration were commissioned from external research providers and most are available on the Commission’s website.

For most of the time during the 11 months between June 2021 and April 2022 no more than six staff were working on the immigration inquiry. Release of wide range of research reports, in addition to an issues paper, a draft report and the final report, is impressive for a small staff.

Satisfaction with the Commission’s management of research processes

Participants in the Commission’s research process surveyed who agreed or strongly agreed that overall they were satisfied with the Commission’s approach:

17.1% strongly agreed

65.8% agreed

The extent to which inquiry issues papers, draft reports and final reports were delivered to schedule

Independent expert reviewer (Professor Buckle) noted:

There is a systematic work planning process in place developed in consultation with the PC Board and other stakeholders. There is now a systematic quality assurance process in place. There seems to be a stronger determination to seek peer review of ERT research through the refereeing of working papers, the presentation of research at academic conferences, and the publication of research beyond the PC website, including in peer reviewed research journals.

Satisfaction with the Commissioner’s management of research processes

Independent expert reviewer (Professor Buckle) noted:

That with the support of the Board and additional financial support, the Commission had been able to “markedly improve the level of research experience, skill and capacity”.

“a research management process has been implemented.”

The degree of reviewer confidence in research findings and conclusions

Independent expert reviewer (Professor Buckle) noted:

There is now a systematic quality assurance process in place. There seems to be a strong determination to seek peer review of ERT research through the freeing of working papers, the presentation of research at academic conferences, and the publication of research beyond the PC website, including peer reviewed research journals.

Confidence in inquiry findings and recommendations

Participant confidence in the Commission’s inquiry findings and recommendations

Inquiry participants surveyed who considered the following aspects to be of good or excellent quality:

The inquiry’s analysis of information

18.5% excellent

43.2% good

The findings and recommendations

12.3% excellent

44.4% good

Inquiry participants surveyed who agreed or strongly agreed that:

The Commission’s recommendations followed logically from the inquiry analysis and findings

19% strongly agreed

58.2% agreed

The Commission’s recommendations would, if implemented, materially improved performance in [the topic area]

18.8% strongly agreed

42.5% agreed

Summary of focus group

Focus group summary of whether they considered the following aspects to be of good or excellent quality:

The inquiry’s analysis of information

Several comments were made on the quality of the data and analysis – one participant talked about how difficult it was to analyse immigration patterns before the Commission put out information and analysis as part of the inquiry.

“We used to be reliant on flow numbers and had to read multiple reports to analyse what was happening in the labour market.”

“The way they analysed immigration by industry was really important for improving understanding.”

However, there were contrary views about quality, with one person saying:

“The Commission has an excellent reputation, so this felt like a lighter and more targeted approach, with a process that was quite different to what we’ve seen previously.”

The findings and recommendations

The recommendations were supported by most participants, with positive comment being made about recommendations specifically addressing:

• the ability of migrant workers to change employer;
• the need for more support for migrant workers as they start work in the NZ environment;
• proposals for more data and labour market modelling.

Focus group summary of whether they agreed or strongly agreed that:

The Commission’s recommendations followed logically from the inquiry analysis and findings

The Commission’s recommendations would, if implemented, materially improve performance in [the topic area]

People were looking for more acknowledgement of, and response to the inquiry from both the Government and MBIE. Decisions on sector workforce agreements and the Green List had cast doubt over how much influence the inquiry would have.

“Immigration NZ isn’t referencing the inquiry.”

Expert review commentary – Richard Bedford

Participant confidence in the Commission’s inquiry findings and recommendations

Inquiry participants surveyed who considered the following aspects to be of good or excellent quality:

The inquiry’s analysis of information

Question not asked of expert reviewer.

The findings and recommendations

Question not asked of expert reviewer.

Inquiry participants surveyed who agreed or strongly agreed that:

The Commission’s recommendations followed logically from the inquiry analysis and findings

Question not asked of expert reviewer.

The Commission’s recommendations would, if implemented, materially improved performance in [the topic area]

Question not asked of expert reviewer.

The degree of reviewer confidence in research findings and conclusions

(The E&R team were embedded in the Immigration inquiry to conduct research for the inquiry. Therefore, the inquiry reviewer also looked at the E&R team’s contribution to this inquiry.)

Reviewer commentary indicates the following aspect to be of good or excellent quality:

• Information analysis or research papers

...I am impressed with the quality of the process that was adopted to meet the terms of reference for the immigration inquiry. I am also impressed with the inquiry’s four major outputs: the Issues paper, the draft report, the final report and the report (Immigration by the numbers) that contains much of the quantitative evidence that has informed the Commission’s findings and recommendations.

The data presented in Immigration by the numbers provides support for several of the Commission’s headline statements about the inquiry that are on their website, including: “On average, immigration is not driving down wages nor displacing local workers.”

Reviewer commentary indicates the following aspect to be of good or excellent quality:

• Findings of research papers

Immigration by the numbers is, and will remain, a major resource for anyone interested in international migration in New Zealand. It is rare to find such a comprehensive and very readable report on immigration in New Zealand.

The Commission’s Economics and Research team are to be commended on the high quality of both the presentation of data in this report.

Immigration – Fit for the future is well written, has a clear and coherent structure, and is well-illustrated. Its content meets the requirements of the Minister of Finance in his letter of expectations in May 2021.

A major contribution of the Commission’s immigration inquiry has been to ‘red flag’ a number of key issues, such as absorptive capacity, that Ministries need to consider and address, rather than trying to do all the thinking for the Ministries.

In this sense, Immigration – Fit for the future and the other reports and working papers that have been produced during the course of the immigration inquiry have made a particularly valuable contribution that extends well beyond the domain of immigration policy settings.

Reviewer agreed or strongly agreed that:

• Conclusions followed from analysis and findings

The data presented in Immigration by the numbers provides support for several of the Commission’s headline statements about the inquiry that are on their website, including: “On average, immigration is not driving down wages nor displacing local workers.”

Confidence in inquiry findings and recommendations

Expert reviewer (Professor Robert Buckle) noted:

The review found that the team “is perhaps the strongest it has been for several years.”

“Despite capacity pressures, the PC and ERT have been active in presenting research papers at local research conferences. This has had the strong support of the PC Commissioners and involvement by one of the Commissioners, Professor Gail Pacheco. The topics of several of these research papers had a clear connection to the work on inquiries at the time.”

For example, in his review of the report, Living on the edge: An anatomy of New Zealand’s most productive firms, Prof Buckle noted:

“This work is of a very high quality. As some policy agency staff have remarked, this is a highly valuable “state of the art” source of information about the productivity of New Zealand firms. The data preparation, analysis and derivation of characteristics are meticulously documented and undertaken with rigour.”

Confidence in inquiry findings and recommendations

Participant perception of the quality of engagement by the Commission.

Inquiry participants surveyed who agreed or strongly agreed that:

In total, 91 people responded to the following statements:

•    There was ample opportunity to participate in the inquiry

37% strongly agreed

45.7% agreed

•    The Commission was approachable

32.2% strongly agreed

44.4% agreed

•    The Commission communicated clearly

28.9% strongly agreed

51.1% agreed

•    The Commission understood their views

21.1% strongly agreed

41.1% agreed

Focus group perception of the quality of engagement by the Commission.

One group member commented that “this Government is not known for the quality of its engagement on policy proposals” but said that the Commission’s work was an exception.

Other feedback included:

“This was the most solid piece of consultation I have seen in a long time.”

“Certainly, the submissions reflected a wide range of organisations.”

People who had been involved in face-to-face and Zoom meetings said the meetings had worked well and that staff at the Commission were both highly competent and good to deal with.

Another person said that his engagement with the Commission over the report had been excellent. Where he had disagreed or didn’t understand something, the Commission did a good job of explaining and taking his input on board.

The Commission’s engagement process was contrasted with the reference groups that are often used to gain stakeholders views. One participant commented that the Commission’s approach brought out the views of members of a range of organisations rather just the views of a few individuals on a reference group.

Expert reviewer (Richard Bedford) comments on perception of the quality of engagement by the Commission.

The Commission’s approach was a positive contribution toward improved levels of coordination and collaboration in productivity research.

The majority (80%) of these engagement meetings were with institutional and corporate stakeholders. Only 15 (20%) were with named academics, policy experts and commentators.

Of the 59 engagement meetings with institutions, organisations, and corporate entities, 18 (30%) involved officials in Government ministries and agencies, 14 (24%) involved members of industry organisations and eight (14%) with members of education institutions/bodies.

More than twice as many institutions, organisations and corporate entities (68) made submissions on the draft report than responded to the call for submissions on the issues paper (30).

“…a high level of engagement with the immigration inquiry from non-government organisations.”

One crude indicator of effective engagement is the extent to which submissions on the draft report, and the insights gained from new research that the Commission had conducted internally or had commissioned externally, resulted in substantive changes in the key messages contained in the final report.

The significant refinement of many of the recommendations that were contained in the draft report, plus the inclusion of 12 new ones in the final report, reflects positively on the effectiveness of the engagement process.

Following a period of robust debate and deliberation, the Commission responded constructively to ideas, criticisms and new evidence contained in submissions and research reports and met the delivery target for its final report at the end of April 2022.

Participants in Commission research processes surveyed who agreed or strongly agreed that the Commission’s approach was a positive contribution in productivity research.

We cannot report against this measure as this question was not included specifically in the survey.

Engagement meetings held

Number of parties the Commission engaged with during the inquiry, including Māori organisations, as noted in the final report appendix.

74 Engagement meetings

Submissions received

Number of parties who made a submission during the inquiry, as noted in the final report appendix.

Understanding of productivity related matters increases

Expert reviewer (Professor Robert Buckle) noted:

“The broadening of engagement initiatives to reach diverse audiences and to enhance research collaboration opportunities. These initiatives range from opinion-editorials (op-eds) in New Zealand newspapers, research conference initiatives, and research collaboration with selected government departments.”

In one of the reports reviewed in depth, Migration and firm-level productivity, by Richard Fabling, David C. Maré and Philip Stevens, Prof Buckle noted:

“This is an excellent and substantial body of research… It is perhaps not surprising that this paper was mentioned by some from government departments as an example of the value the E&R team can provide to agencies that do not have the resources to undertake data intensive research that could add rigour to policy advice to Ministers.”

Perception of how well our work was communicated and presented

Participant survey

Participant perception of the effectiveness of the Commission’s communication of inquiry and research findings and recommendations.

Inquiry participants surveyed who agreed or strongly agreed that:

The findings and recommendations were clear.

28.7% strongly agreed

56.3% agreed

The style of writing and language used in the report was clear.

36.7% strongly agreed

54.4% agreed

The summary material provided was useful (the 100- page format was concise and easily digestible).

26.3% strongly agreed

46.3% agreed

Focus group commentary of inquiry and research papers indicates that:

(The E&R team were embedded in the Immigration inquiry to conduct research for the inquiry. Therefore, the inquiry reviewer also looked at the E&R team’s contribution to this inquiry.)

The conclusions were clear.

In terms of the individual recommendations, there was a lot of support for the proposal for a Government Policy Statement. A number of people felt that the Statement would make policy transparent when immigration has been such a closed- door business, and provide a concrete basis for challenging what is happening on the ground:

“The Government Policy Statement recommendation is really important to get immigration policy out in the open.”

The style of writing and language used was clear.

Most participants found the overview and Immigration by the numbers very useful. People said that the final report was of very good quality: well set-out, made good use of boxes, graphs, charts and data, and was easy to read and navigate.

“I enjoyed reading the final report. It was fantastic to have the numbers separate and to see all the views of interested parties.”

However, some people had been looking for the Commission to propose a major overhaul of the immigration settings:

“With the borders closed, this was a critical moment to take a good hard look at an immigration system that is broken.”

“While the recommendations were really useful and important, they lacked a compelling, driving single argument for a major overhaul.”

They thought that the Commission had done a reasonable job, but it was hampered by a narrow interpretation of the Terms of Reference, internal staff issues, parallel policy processes, and Covid.

The summary material was useful.

The focus groups were not asked this specific question.

Reviewer commentary of research papers indicates that:

The conclusions were clear.

The Productivity Commission’s immigration inquiry has scored highly on all of the performance measures that this review has been asked to consider. The Commission did a remarkable job delivering on the very ambitious ToR, especially given the transition that it was undergoing during the year when all phases of the inquiry were completed.

The style of writing and language used was clear.

Immigration – Fit for the future is well- written, has a clear and coherent structure, and is well-illustrated. Its content meets the requirements of the Minister of Finance in his letter of expectations in May 2021 where he requests that the Commission:

“looks beyond traditional measures of economic success such as GDP, and has the wellbeing of current and future generations of New Zealanders front of mind as it generates new knowledge and advice.”

Paper(s) provided clarity about steps leading on from the research.

To ensure that immigration policy is “fit for the future”, the Commission argues that a range of absorptive capacity issues must be addressed, along with how best to reflect Te Tiriti o Waitangi in immigration policy and institutions.

To achieve this, the Commission recommends establishing an immigration Government Policy Statement (GPS) to improve the quality and transparency of immigration policy.

The summary material was useful

The expert reviewer was not asked to comment specifically on this.

Perception of how well our work was communicated and presented

Expert reviewer (Professor Robert Buckle) noted:

While generally positive, economists and researchers interviewed for the report suggested more could be done by to design documents that complement the more technical research papers, and to draw out the policy insights.

In his review of the Productivity by the numbers series, Prof Buckle noted:

“The second Productivity by the numbers publication released in 2021 and the new Immigration by the numbers published in 2022

… present chapters on the history of immigration and productivity performance in New Zealand, a non-technical discussion on the way immigration and productivity contribute to firm performance, labour markets, economic performance and wellbeing. The 2021 version of Productivity by the numbers also contains a chapter discussing what governments can do to improve productivity.

“People interviewed indicated strong support for the … reports and appreciation of the information they provide. There were particularly appreciative comments on the style and quality of the 2021 publication and that it was a valuable source document on New Zealand productivity matters.”

Perception of overall quality

Independent expert evaluation of the overall quality of the inquiry.

Independent expert reviewer (Richard Bedford) noted:

A report evaluating the overall performance of the inquiry from the final inquiry report (taking into account the focus of the report, process, analysis, engagement and delivery of message) with recommendations of future improvements.

The Productivity Commission’s immigration inquiry has scored highly on all of the performance measures that this review has been asked to consider: the inquiry process, engagement strategies, focus of the inquiry, quality of the research and reports, and delivery of key messages. I have not reviewed other inquiries undertaken by the Commission so I cannot make any comments about how this inquiry compares with others on the basis of these performance criteria. But as someone who has organised multi-year research programmes on immigration and population change in New Zealand, I have been impressed by the scale, scope and productivity of the Commission’s immigration inquiry.

On balance, I rate the overall quality of the inquiry as being very high. There will be on-going debate, amongst economists especially, about some of its findings. That is not a negative sign, especially in the social sciences. There will be debate about several of the recommendations. That is a positive sign that the Commission’s findings are challenging the status quo and triggering discussion about issues people consider to be important. There are some gaps in the inquiry, and some omissions from the engagement list, that I became aware of during the course of the review. These are inevitable in such a wide-ranging inquiry into a topic around which there is considerable debate.

My only suggestion for future inquiries that address topics rooted in demography is to make sure that the demographers are consulted. I was surprised to find that one of New Zealand’s most experienced demographers, Dr Natalie Jackson, who was probably done more work on contemporary demography of New Zealand at national as well as regional levels, was not consulted at any stage during the inquiry. Economists, sociologists and geographers who work with population data are not, by definition, demographers.

Focus group evaluation of the inquiry

Report from a focus group representative of inquiry participants facilitated by an independent person with significant experience in inquiry- type work (taking into account the focus of the report, process, analysis, engagement and delivery of message) with feedback on the inquiry and recommendations for future improvement.

Many participants thought the quality of the Commission’s work was very high overall, including the preliminary and final reports, other supporting material, the process and the engagement.

Comments included:

“A very robust piece of work.”

However, some people had been looking for the Commission to propose a major overhaul of the immigration settings:

“With the borders closed, this was a critical moment to take a good hard look at an immigration system that is broken.”

“While the recommendations were really useful and important, they lacked a compelling, driving single argument for a major overhaul.”

The group thought that the Commission had done a reasonable job, but it was hampered by a narrow interpretation of the terms of reference, internal staff issues, parallel policy processes, and COVID-19.

One participant suggested it would be useful for the Commission undertake a second immigration inquiry, to look at the labour market impacts of recent immigration policy changes, and the opening of the border, over the coming two years. It could also analyse what happens to migrant workers six months and two years post-immigration.

Several people, including those who had been somewhat disappointed with this inquiry, voiced strong support for the ongoing work of the Commission.

“Keep on consulting us and other stakeholders. We fully support the Commission’s work.”

Participant evaluation of the inquiry

Percentage of inquiry participants surveyed who rated the overall quality of the inquiry as good or excellent (taking into account the focus of the report, process, analysis, engagement and delivery of message).

17.8% excellent

42.5% good

26% acceptable

[Note: Participants who rated the overall quality of the inquiry positively, as acceptable, good or excellent (a less demanding standard than the performance measure)]

Perception of overall quality

Expert reviewer (Professor Robert Buckle) noted:

Prof Buckle noted the rebuilding of ERT and PC productivity research leadership and capability was both timely and positive, and “is taking place at a time when productivity issues seem to be regaining attention”.

“Discussions with those interviewed during this review emphasised the value of the research emerging from ERT and the insights the research provided for policy agencies.”