How And Why Artificial Intelligence Is Being Used To Process Your Submissions To Politicians

Scoop

6 days ago

Explainer - The public likes to have their say.
Tens of thousands of public submissions come in every year to bills before Parliament and to local government entities.
With large-scale campaigns and website submission forms, the ability to speak out is easier than ever - but that's causing a problem on the other end of the system, where planners and politicians can struggle to keep up.
Artificial intelligence has increasingly been drafted to go over public submissions. Some have applauded the technology's ability to process data quicker than humans, while others fear the human touch may be getting lost in the shuffle.
What exactly does AI processing of public submissions mean, how does it work, and are everyone's views getting a fair shake in the process?
Here's a breakdown of it all.
First, how do public submissions work?
It's a chance for people to get their voice heard in local and national government.
People can make submissions to both their local councils and to Parliament. Submissions can be made to local councils on things like planning and urban development, while the public can make submissions to Parliament select committees on upcoming bills.
Submissions have been sky-high in recent months, where the Treaty Principles Bill received more than 300,000 submissions, while the Regulatory Standards Bill which is now before Parliament also has had huge interest. Final submission numbers on that have not been released, but even the early discussion on the proposed bill at the end of last year received about 23,000 submissions.
Dr David Wilson, Clerk of the House of Representatives who oversees the business of Parliament's rules and procedures, said public input is at a high.
"The Treaty Principles Bill had more submissions than the last two parliaments combined," he said. At one point submission numbers were so large the website suffered technical difficulties.
Wilson said the number of submissions does put a strain on resources in Parliament.
"If that is the sorts of volumes we're going to see on more and more bills, the days of human beings being able to deal with them in a sort of reasonable time will be past."
When submissions come to Parliament, staff of the Office of the Clerk first process them to make sure they are relevant to the bill and not defamatory or insulting before they go on to select committees.
Select committees then process and consider feedback before making possible changes to a bill ahead of a final vote on it.
"It's great that the public want to engage with Parliament and see the value in making their thoughts known even in such volumes," Wilson said.
"I think people understand that no individual MP could read 300,000 submissions. We can't create more time for MPs to read them."
Eddie Clark, a senior lecturer in public law at Victoria University of Wellington who is critical of AI use in public submissions, noted that large numbers of submissions have been processed before AI became widely available, such as the Conversion Practices Prohibition Legislation Bill in 2021 which received more than 100,000 entries.
"So it is possible for very large numbers of submissions to be actually read and processed by actual human staff. What was required was time and resource, and in my opinion the denial of both is a reason the huge number of submissions has become such a problem several times over the last couple of years."
Enter artificial intelligence
This is where artificial intelligence is starting to come in - both in local and national government, where it's being used to help process, sort and analyse public input.
The Office of the Clerk does not use AI in processing submissions, but it's up to the individual committee overseeing the bill to decide whether to do so when the bills come to their end, Wilson said. For instance, it's been used along the way for the Regulatory Standards Bill.
"Committees make their own individual decisions; they don't have any central guidelines around it at the moment."
Wilson said the Office of the Clerk is looking at how it might use AI in the future, but is being cautious and "not rushing into it".
"I still think ultimately we need to have human decision makers but AI has capacity to do things more quickly than people can - such as flagging submissions that are irrelevant or defamatory. Most submissions are absolutely fine."
AI processing has been taken up by local councils, too.
In Nelson, the city council worked with local firm the AI Factory to process submissions to their long term plan, Group Manager for Strategy and Communications Nicky McDonald said.
"We used the tool to analyse views on issues, including numbers for/against, and to provide us with a summary of views which we then used when writing the first draft of our deliberations report to council.
"This report went through multiple iterations as we edited it, but AI was able to give us a starting point which we then developed into a final draft."
Xinyu Fu, a senior lecturer in environmental planning at the University of Waikato, organised a pilot project with Hamilton City Council analysing thousands of public submissions on planning proposals.
"A lot of them are facing stresses on analysing public submissions," he said of local planners.
"Planners spend a lot of time going through those public submissions and those are very laborious work."
What exactly are they using AI to do?
Prompts - instructions, questions and information put into generative AI - are used to direct it.
In Hamilton, Fu's research paper explained that "we tasked ChatGPT with extracting five key elements from public feedback: 1) political stance (support, opposition, or unspecified), 2) reasons from submitters, 3) decisions sought by submitters, 4) sentiment of the submission (positive, negative, or neutral), and 5) relevant planning topics."
"AI models are sensitive to prompt phrasing so a slight change in prompt may result in changes in its responses," Fu said.
With the Regulatory Standards Bill, public feedback on the discussion document last year drew 22,821 submissions.
(The feedback to the select committee on the bill itself is still being processed and is confidential until the Finance and Select Committee releases that information.)
In a summary of submissions, the Ministry for Regulation said that all submissions on the then-proposed bill were analysed using a Large Language Model (LLM) AI, and it worked with the independent research organisation Public Voice.
"All emails and Citizen Space submissions (a digital tool that submits an online form) were assigned a preliminary classification by Public Voice using a LLM that followed a logic model created by the Ministry, analysing it and classifying it as supporting, partially supporting, opposing the bill or unclear on its stance."
The majority of submissions on the proposed bill were analysed by AI.
However, the summary also said that in a qualitative analysis sample, 939 of those 22,821 submissions were examined by Ministry for Regulation staff to "analyse the themes raised in submissions and feedback on specific policy proposals." That process "involved several staff across the Ministry manually reviewing the sample of submissions (both email and Citizen space submissions) and applying thematic tags."
Another 605 submissions were also looked at separately. Submissions made in te reo Māori were translated.
"Our approach was carefully designed to reflect all submissions in the final analysis, noting there were many similar points made across most of the submissions," the ministry's deputy chief executive Andrew Royle told Newsroom.
How much human scrutiny is applied to the process? Can the AI avoid a bias?
"As a rule of thumb, having humans in the loop will be the best practice - humans in charge and AI as a co-pilot," Fu said.
"The risk is very high if we completely rely on AI to do the work. To put simply, such biases are generally embedded in our institutions as well as the information humans generated, and these biases are then input into the model to train. Then they become inherent to the model. Because AI systems are black boxes, it is uncertain and unclear about the nature and degree of these biases."
Nelson Council's McDonald said they were transparent about how they were using AI.
"Every submission form included a statement saying we'd be trialling AI to help speed up submission processing and reduce the resource burden on staff.
"We intentionally ensured there was always a (sceptical!) human in the loop sense checking the tool's outputs. Staff (and elected members) read every submission and we had processes to check AI responses."
Fu said there are differences in how AI approaches looking at thousands of public submissions.
"AI is really good at consistency (if instructed properly) whereas humans are likely to miss things due to fatigue, boredom, or bias towards particular viewpoints (humans are biased too).
"AI can do things much faster than humans, and AI's work can be more transparent if designed well because you can ask AI to document its processes and responses for later review and replication. On the downside, humans excel in knowing about the contexts, while AI knows little about the local contexts and backgrounds."
Is there a risk that people's voices aren't being heard?
"I absolutely think that a regular practice of AI analysis of submissions risks undermining people's confidence in the democratic process and thus the legitimacy of government," Victoria University's Clark said.
He said there was a need for more options for people to consult on legislation. He noted in the case of the Regulatory Standards Bill, the pre-legislative consultation was conducted mostly over the holiday period from mid-November to mid-January.
This "leads to people seeing the Select Committee stage as their only real chance to comment, incentivising mass submissions expressing simple opposition or support", Clark said.
"Giving people a chance to be heard throughout the process, not just at Select Committee, could help deal with the problem. There is a reason the legislative process is generally slow and deliberate, and derailing that good, democratic process has consequences. In my opinion the glut of submissions at the Select Committee stage is one of them."
Labour MP Duncan Webb spoke out about the government's use of AI on the Regulatory Standards Bill submissions, writing on social media site BlueSky that it "turns out democracy under this government is real people making submissions and computers reading them".
When contacted by RNZ, Webb said he is not opposed to the use of AI, but concerned about how it is used in the democratic process.
"New Zealanders who take the time to share their views deserve more than a computer reading their submission.
"AI can help with sorting large volumes of submissions, but it can't replace the value of reading someone's views, like the handwritten letter from an 85-year-old or a bundle of colourful drawings from school kids. These submissions often reflect deeply held experiences and emotions, and politicians need to read them."
However, Fu said that in local government planning the use of AI in analysis could give staff more time to work with local and underrepresented communities.
"Planning has become very reactive," he said. "If we can use AI planners then planners can actually do better work because otherwise they're overwhelmed."
A lot of the submissions made on local planning tend to be by developers, Fu said. He said planners could use the time to reach out to communities whose voices aren't heard as often in public submissions, including Māori.
What about privacy?
When it comes to privacy, public submissions are already just that - public.
All submissions sent to select committees become public and are posted on Parliament's website and become part of the permanent parliamentary record - they can only be removed in exceptional circumstances by the Clerk of the House.
"They know their submission will become public," Wilson said of submissions. "Our staff are going to read it, officials will read it."
"The main privacy concern is about people's contact details - they are always separated from submissions now."
Contact information is removed from public submissions before they are posted publicly but Wilson said privacy is one reason to be cautious of the use of AI in analysing them.
"We want to make sure we've got a key set of principles and some business rules in place," Wilson said.
The government unveiled its first national AI strategy earlier this month mostly aimed at economic growth, "unlocking innovation, productivity, and smarter decision-making across New Zealand" and responsible AI guidance for businesses "to overcome concerns about ethics and complexity."
In Nelson, McDonald said they also considered privacy issues.
"The submissions, numbering 1505, were redacted of all personal data before they were processed to ensure there were no privacy issues - this is something we would do anyway, before all submissions are uploaded to the Council website for public view."
Where should AI not be used?
Most agree AI should never be making decisions on policy, however.
"What I don't think I can do - and I wouldn't trust it to do anyway - is make judgements," Wilson said.
"Nobody's going to predict what's going to happen next month in the AI space because it's evolving so rapidly," Fu said, noting that hyperbole over AI is everywhere at the moment.
"We're still in that hype space ... I think we need to start thinking about the responsible use."
And for some, there's still a question as to whether the technological advances of AI might be leaving something behind.
"In short, democracy takes money and time," Clark said.
"Trying to avoid the necessary costs of democratic infrastructure has consequences, and while I understand why the hard-working people in our underfunded and rushed systems might see AI as helpful in these circumstances, in my opinion it will not solve the underlying issue and could unintentionally undermine people's faith in a democracy that cares about their voices."