Latest news with #MultistateBarExamination
Los Angeles Times
02-05-2025
- Politics
- Los Angeles Times
Head of State Bar of California to step down after exam fiasco
The State Bar of California announced Friday that its embattled leader, who has faced growing pressure to resign over the botched February roll out of a new bar exam, will step down in July. Leah T. Wilson, the agency's executive director, informed the Board of Trustees she will not seek another term in the position she has held on and off since 2017. She also apologized for her role in the February bar exam chaos. 'Accountability is a bedrock principle for any leader,' Wilson said in a statement. 'At the end of the day, I am responsible for everything that occurs within the organization. Despite our best intentions, the experiences of applicants for the February Bar Exam simply were unacceptable, and I fully recognize the frustration and stress this experience caused. While there are no words to assuage those emotions, I do sincerely apologize.' Wilson's last day will be July 7. Many February test takers urged Wilson to resign after the exam, which critics say was rolled out hastily in a bid to save money and ultimately plagued by technical glitches and irregularities. Last week, further controversy erupted when it became clear that the State Bar had not been transparent about the use of artificial intelligence to develop multiple-choice questions. The news of Wilson's departure came on the day that thousands of February test takers were scheduled to get their exam results. But the results appear to have been delayed after the State Bar was late filing a petition with the California Supreme Court on scoring adjustments for the exam that also responded to the court's questions about how and why it utilized AI to develop multiple-choice questions. The State Bar filed a petition to the Supreme Court on Tuesday — and test takers remain in limbo, unsure when they will learn if they passed or failed. Since the debacle, Sen. Tom Umberg (D-Orange), chair of the state Senate Judiciary Committee, and many legal experts have called on the State Bar to ditch the new questions and revert to the traditional test format in July — at least until new questions and methods are adequately tested. On April 25, deans at more than a dozen California's American Bar Association-accredited law schools wrote to Patricia Guerrero, chief justice of the California Supreme Court, expressing 'serious concerns about the exam's fairness and validity.' The deans urged the court to release all 200 multiple-choice questions that were on the February exam and return to using the NCBE's Multistate Bar Examination for the multiple-choice portion of the next exam. Wilson, however, signaled Friday that California should ultimately push ahead with its own bar exam. 'As the fourth largest economy in the world, it is only right that California develops its own bar exam, and that ultimately that exam reflect the innovation, excellence, equity, and accessibility principles that are central to who we are as Californians,' she said in a statement. 'We will not get there by turning backward.' Wilson first took on the role of executive director in 2017, but exited briefly to work for a consulting firm before returning in 2021. She faced additional scrutiny for her income — she earned an annual 2023 salary of $362,067, plus $59,968 in bonuses —at a time when the State Bar is struggling financially. 'Stagnating revenue and increasing personnel costs,' California's state auditor said in a recent report, 'have led its general fund to a deficit in four of the last five years.' Still, Wilson said she was proud of her time at the helm of the State Bar, citing her efforts to make the organization an 'exceptional workplace' that resulted in 'strong staff engagement, positive union relationships and historically low turnover.' 'Over the course of nearly 10 years, I have had the privilege of leading the State Bar through a period of transformative change,' Wilson said in a statement. 'I am particularly proud of our efforts to elevate and offer real solutions to the access-to-justice crisis in our state, make real our commitments to increasing equity and inclusion in the profession, and stabilizing the State Bar financially,' Wilson added. Brandon Stallings, chair of the State Bar's Board of Trustees, praised Wilson's leadership, noting she had played a key role in advancing many of the organization's critical initiatives. 'The Board recognizes the significant contributions that Leah Wilson made during her tenure, particularly in the concerted effort to recognize and address racial disparities in the discipline system,' Stallings said. 'We understand and respect her decision, and we are grateful for her service.'
Yahoo
26-04-2025
- Business
- Yahoo
Pressure grows on California State Bar to revert to national exam format in July after botched exam
An influential California legislator is pressuring the State Bar of California to ditch its new multiple-choice questions after a February bar exam debacle and revert to the traditional test format in July. 'Given the catastrophe of the February bar, I think that going back to the methods that have been used for the last 50 years — until we can adequately test what new methods may be employed — is the appropriate way to go,' Sen. Thomas J. Umberg, chair of the state Senate Judiciary Committee, told The Times. Thousands of test takers seeking to practice law in California typically take the two-day bar exam in July. Reverting to the national system by the National Conference of Bar Examiners, which California has used since 1972, would be a major retreat for the embattled State Bar. Its new exam was rolled out this year as a cost-cutting measure and "historic agreement" that would offer test takers the choice of remote testing. Alex Chan, an attorney who chairs the Committee of Bar Examiners, which exercises oversight over the California bar exam, told The Times earlier this week it was unlikely that the State Bar would revert to the NCBE exams in July. 'We're not going back to NCBE — at least in the near term,' Chan said. The NCBE's exam security would not allow any form of remote testing, Chan said, and the State Bar's recent surveys showed almost half of California bar applicants want to keep the remote option. Read more: State Bar of California admits it used AI to develop exam questions, triggering new furor Last year, the financially strapped State Bar made the decision to cut costs by replacing the test questions developed by the National Conference of Bar Examiners' Multistate Bar Examination, which does not allow remote testing. If the State Bar developed its own questions, it figured it could save money by sparing the expense of renting massive exam halls for all test takers. The State Bar hired a vendor, Meazure Learning, to administer the exam and announced an additional $8.25-million five-year deal authorizing test prep company Kaplan Exam Services to create multiple-choice, essays and performance test questions. But after the botched rollout of the new exam in February — when many test takers complained of a litany of technical problems, glitches and irregularities — the state's highest court, which oversees the State Bar, directed the agency to plan on administering the July exam in the traditional in-person format. The Supreme Court has yet to direct the State Bar to return to the NCBE system, even though test takers complained that some of the multiple-choice questions in the new test included typos and questions with more than two correct answers and left out important facts. This week, the State Bar enraged test takers and legal experts when it revealed that it had hired ACS Ventures, its independent state psychometrician that validates and scores exams to ensure they are reliable, to develop a small subset of multiple-choice questions using artificial intelligence. 'They have to go back to the multi-state bar exam this summer,' said Katie Moran, an associate professor at the University of San Francisco School of Law. 'They have just shown that they cannot make a fair test.' Erwin Chemerinsky, dean of the UC Berkeley Law School, agreed. 'The reality is that remote options, as shown in February, work poorly," he said. "The bar exam is too important for them to experiment as they did and are continuing to do.' State Sen. Umberg, a former prosecutor, likened having a non-lawyer using artificial intelligence to draft questions for a bar exam "to non-physicians designing questions with the help of AI to decide who's qualified to be a surgeon.' Read more: 'Utterly Botched': Glitchy rollout of new California bar exam prompts lawsuit and legislative review As chair of the Senate Judiciary Committee, Umberg wields considerable clout over the State Bar. He recently pushed Senate Bill 40, a new law that requires the state Senate to confirm future appointments of the State Bar's executive director and general counsel. After the February exam debacle, Umberg filed legislation to launch an independent review of the exam by the California State Auditor, to find out what went so 'spectacularly wrong.' That bill is slated to be reviewed at a May 6 Senate Judiciary Committee hearing, along with Senate Bill 253, the State Bar's annual license fee authorization bill, which gives lawmakers leverage to push the Bar to make improvements. Umberg said the upcoming hearing will go beyond just the administration of the February bar exam. 'We're going to be looking at the leadership of the bar,' he said. 'We're going to be looking at what's happened since our last oversight hearing in terms of accountability and transparency.' Asked whether he had confidence in State Bar leadership. Umberg said: 'My confidence is shaken.' Umberg would not say whether the State Bar's executive director Leah Wilson should step down, but said the question is 'one of the issues that we'll be examining here in the months ahead.' The State Bar announced this week it will ask the Supreme Court to adjust test scores for those who took its February bar exam. For critics of the State Bar, the problem is not just that it used AI to develop questions, but that it did so without the knowledge of the California Supreme Court and the Committee of Bar Examiners. Read more: California Supreme Court demands State Bar answer questions on AI exam controversy The State Bar told The Times the decision to have ACS Ventures develop questions with the assistance of AI programs 'was made by staff within the Admissions office and not clearly communicated to State Bar leadership." 'This was a breakdown, and structural changes have been made within Admissions to address it,' the State Bar said, noting that it has since created a new chief-level role over Admissions reporting directly to the Executive Director and "a new team structure to strengthen accountability and effectiveness.' At the same time, the State Bar downplayed the significance of hiring ACS Ventures to develop questions, noting the company's 'general support of the bar exam — of which the CBE and Board are aware — is covered by their existing contract.' All multiple-choice questions, including those 'developed initially with the assistance of AI,' the State Bar said, 'were subsequently reviewed by content validation panels comprised of lawyers, and an attorney subject matter expert, as part of the question development and finalization process.' Whatever happens next, Umberg said, the State Bar should take the time to prevent the debacle of the February bar exam from happening again. 'Taking the bar exam, it's really a test that people prepare for for three or more years,' Umberg said, noting two of his children had taken it in recent years. 'The fact that the test takers, in essence, were guinea pigs for the February bar is absolutely unacceptable.' 'That's why," Umberg said, "we're going to go back to the old methodology here in July." Sign up for Essential California for news, features and recommendations from the L.A. Times and beyond in your inbox six days a week. This story originally appeared in Los Angeles Times.
Los Angeles Times
26-04-2025
- Business
- Los Angeles Times
Pressure grows on California State Bar to revert to national exam format in July after botched exam
An influential California legislator is pressuring the State Bar of California to ditch its new multiple-choice questions after a February bar exam debacle and revert to the traditional test format in July. 'Given the catastrophe of the February bar, I think that going back to the methods that have been used for the last 50 years — until we can adequately test what new methods may be employed — is the appropriate way to go,' Sen. Thomas J. Umberg, chair of the state Senate Judiciary Committee, told The Times. Thousands of test takers seeking to practice law in California typically take the two-day bar exam in July. Reverting to the national system by the National Conference of Bar Examiners, which California has used since 1972, would be a major retreat for the embattled State Bar. Its new exam was rolled out this year as a cost-cutting measure and 'historic agreement' that would offer test takers the choice of remote testing. Alex Chan, an attorney who chairs the Committee of Bar Examiners, which exercises oversight over the California bar exam, told The Times earlier this week it was unlikely that the State Bar would revert to the NCBE exams in July. 'We're not going back to NCBE — at least in the near term,' Chan said. The NCBE's exam security would not allow any form of remote testing, Chan said, and the State Bar's recent surveys showed almost half of California bar applicants want to keep the remote option. Last year, the financially strapped State Bar made the decision to cut costs by replacing the test questions developed by the National Conference of Bar Examiners' Multistate Bar Examination, which does not allow remote testing. If the State Bar developed its own questions, it figured it could save money by sparing the expense of renting massive exam halls for all test takers. The State Bar hired a vendor, Meazure Learning, to administer the exam and announced an additional $8.25-million five-year deal authorizing test prep company Kaplan Exam Services to create multiple-choice, essays and performance test questions. But after the botched rollout of the new exam in February — when many test takers complained of a litany of technical problems, glitches and irregularities — the state's highest court, which oversees the State Bar, directed the agency to plan on administering the July exam in the traditional in-person format. The Supreme Court has yet to direct the State Bar to return to the NCBE system, even though test takers complained that some of the multiple-choice questions in the new test included typos and questions with more than two correct answers and left out important facts. This week, the State Bar enraged test takers and legal experts when it revealed that it had hired ACS Ventures, its independent state psychometrician that validates and scores exams to ensure they are reliable, to develop a small subset of multiple-choice questions using artificial intelligence. 'They have to go back to the multi-state bar exam this summer,' said Katie Moran, an associate professor at the University of San Francisco School of Law. 'They have just shown that they cannot make a fair test.' Erwin Chemerinsky, dean of the UC Berkeley Law School, agreed. 'The reality is that remote options, as shown in February, work poorly,' he said. 'The bar exam is too important for them to experiment as they did and are continuing to do.' State Sen. Umberg, a former prosecutor, likened having a non-lawyer using artificial intelligence to draft questions for a bar exam 'to non-physicians designing questions with the help of AI to decide who's qualified to be a surgeon.' As chair of the Senate Judiciary Committee, Umberg wields considerable clout over the State Bar. He recently pushed Senate Bill 40, a new law that requires the state Senate to confirm future appointments of the State Bar's executive director and general counsel. After the February exam debacle, Umberg filed legislation to launch an independent review of the exam by the California State Auditor, to find out what went so 'spectacularly wrong.' That bill is slated to be reviewed at a May 6 Senate Judiciary Committee hearing, along with Senate Bill 253, the State Bar's annual license fee authorization bill, which gives lawmakers leverage to push the Bar to make improvements. Umberg said the upcoming hearing will go beyond just the administration of the February bar exam. 'We're going to be looking at the leadership of the bar,' he said. 'We're going to be looking at what's happened since our last oversight hearing in terms of accountability and transparency.' Asked whether he had confidence in State Bar leadership. Umberg said: 'My confidence is shaken.' Umberg would not say whether the State Bar's executive director Leah Wilson should step down, but said the question is 'one of the issues that we'll be examining here in the months ahead.' The State Bar announced this week it will ask the Supreme Court to adjust test scores for those who took its February bar exam. For critics of the State Bar, the problem is not just that it used AI to develop questions, but that it did so without the knowledge of the California Supreme Court and the Committee of Bar Examiners. The State Bar told The Times the decision to have ACS Ventures develop questions with the assistance of AI programs 'was made by staff within the Admissions office and not clearly communicated to State Bar leadership.' 'This was a breakdown, and structural changes have been made within Admissions to address it,' the State Bar said, noting that it has since created a new chief-level role over Admissions reporting directly to the Executive Director and 'a new team structure to strengthen accountability and effectiveness.' At the same time, the State Bar downplayed the significance of hiring ACS Ventures to develop questions, noting the company's 'general support of the bar exam — of which the CBE and Board are aware — is covered by their existing contract.' All multiple-choice questions, including those 'developed initially with the assistance of AI,' the State Bar said, 'were subsequently reviewed by content validation panels comprised of lawyers, and an attorney subject matter expert, as part of the question development and finalization process.' Whatever happens next, Umberg said, the State Bar should take the time to prevent the debacle of the February bar exam from happening again. 'Taking the bar exam, it's really a test that people prepare for for three or more years,' Umberg said, noting two of his children had taken it in recent years. 'The fact that the test takers, in essence, were guinea pigs for the February bar is absolutely unacceptable.' 'That's why,' Umberg said, 'we're going to go back to the old methodology here in July.'
Yahoo
25-04-2025
- Business
- Yahoo
California Admits AI Was Used to Write Bar Exam Plagued by Problems
Now here's a legal-drama-worthy twist in the recent spate of dumb lawyers getting caught using AI: it turns out that the very bar exam administered to aspiring attorneys in California was itself created with the help of a large language model, The Los Angeles Times reports. The admission was made by the State Bar of California on Monday, following complaints about the quality of the test's questions, and numerous glitches experienced by test-takers when they took it in February. In a news release, the organization said that 23 of the exam's total of 171 scored multiple-choice questions were drafted by the firm ACS Ventures, which developed the questions "with the assistance of AI." Another 48 questions were lifted from an older version of an exam for first-year law students. "The debacle that was the February 2025 bar exam is worse than we imagined," Mary Basick, assistant dean of academic skills at UC Irvine Law School, told the LA Times. "I'm almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable." Katie Moran, an associate professor at the University of San Francisco School of Law, called it a "staggering admission." The same company that used AI to draft the questions was then paid "to assess and ultimately approve of the questions on the exam, including the questions the company authored," she noted to the newspaper. For weeks, test takers had complained that they were randomly kicked off the online platform that the bar was administered on, while screens lagged and showed error messages, per the reporting. The test itself was riddled with typos, and some questions were total nonsense. Despite these complaints — and despite pleading guilty to AI usage — a spokesperson for the State Bar insisted that the test questions were reviewed by content validation panels and subject matter experts. In any case, the whole situation sounds like a mortifying catastrophe. For one, the Supreme Court of California, of which the State Bar is an administrative arm, maintains it had no idea about the use of AI to create the test questions until this week — even though it had instructed the State Bar to explore the use of AI to "improve upon the reliability and cost-effectiveness of such testing" last fall, according to Alex Chan, chair of the State Bar's Committee of Bar Examiners. Casting additional scrutiny, Basick and Moran argued that the exam questions, which should take years to develop, were drafted far too quickly, while 50 practice questions re-released just weeks before the actual exam contained numerous errors, they wrote early this month, per the LA Times. What spurred the dubious measures sounds like a familiar tale of disastrous cost-cutting. Faced with a $22 million deficit last year, the State Bar ditched the commonly used National Conference of Bar Examiners' Multistate Bar Examination, and decided to transition to a hybrid model of in-person and remote testing. To create the new test, it inked a $8.25 million deal with Kaplan Exam Services, and contracted Meazure Learning to administer it. In a fittingly legal result, Meazure Learning is now being sued by some of the students who took the glitchy exams. The State Bar said it will ask the California Supreme Court to adjust test scores for those who took the test in February. Chan said that the Committee of Bar Examiners will meet on May 5 to discuss other remedies, but doubted that the State Bar would release the exam questions to the public or go back to the NCBE. More on AI: Judge Goes Ballistic When Man Shows AI-Generated Video in Court
Yahoo
24-04-2025
- Business
- Yahoo
California Supreme Court demands State Bar answer questions on AI exam controversy
The California Supreme Court urged the State Bar of California Thursday to explain how and why it utilized artificial intelligence to develop multiple-choice questions for its botched February bar exams. California's highest court, which oversees the State Bar, disclosed Tuesday that its justices were not informed before the exam that the State Bar had allowed its independent psychometrician to use AI to develop a small subset of questions. The Court on Thursday upped its public pressure on the State Bar, demanding it explain how it used AI to develop questions — and what actions it took to ensure the reliability of the questions. The demand comes as the State Bar petitions the court to adjust test scores for hundreds of prospective California lawyers who complained of multiple technical problems and irregularities during the February exams. Read more: State Bar of California admits it used AI to develop exam questions, triggering new furor The controversy is about more than the State Bar's use of artificial intelligence per se. It's about how the State Bar used AI to develop questions — and how rigorous its vetting process was — for a high stakes exam that determines whether thousands of aspiring attorneys can practice law in California each year. It also raises questions about how transparent State Bar officials were as they sought to ditch the National Conference of Bar Examiners' Multistate Bar Examination — a system used by most states — and roll out a new hybrid model of in-person and remote testing in an effort to cut costs. In a statement Thursday, the Supreme Court said it was seeking answers as to "how and why AI was used to draft, revise, or otherwise develop certain multiple-choice questions, efforts taken to ensure the reliability of the AI-assisted multiple-choice questions before they were administered, the reliability of the AI-assisted multiple-choice questions, whether any multiple-choice questions were removed from scoring because they were determined to be unreliable, and the reliability of the remaining multiple-choice questions used for scoring." Last year, the Court approved the State Bar's plan to forge an $8.25 million, five-year deal with Kaplan to create 200 test questions for a new exam. The State Bar also hired a separate company, Meazure Learning, to administer the exam. It was not until this week — nearly two months after the exam — that the State Bar revealed in a news release that it had deviated from its plan to use Kaplan Exam Services to write all the multiple-choice questions. In a presentation, the State Bar revealed that 100 of the 171 scored multiple-choice questions were made by Kaplan and 48 were drawn from a first-year law students exam. A smaller subset of 23 scored questions were made by ACS Ventures, the State Bar's psychometrician, and developed with artificial intelligence. 'We have confidence in the validity of the [multiple-choice questions] to accurately and fairly assess the legal competence of test-takers,' Leah Wilson, the State Bar's executive director, said in a statement. Read more: Trump releases new files on RFK assassination Alex Chan, an attorney who chairs the Committee of Bar Examiners, which exercises oversight over the California Bar Examination, told The Times Tuesday that only a small subset of questions used AI — and not necessarily to create the questions. Chan also noted that the California Supreme Court urged the State Bar in October to review 'the availability of any new technologies, such as artificial intelligence, that might innovate and improve upon the reliability and cost-effectiveness of such testing.' 'The court has given its guidance to consider the use of AI, and that's exactly what we're going to do,' Chan said. That process, Chan later explained, would be subject to the Court's review and approval. On Thursday Chan revealed to The Times that State Bar officials had not told the Committee of Bar Examiners ahead of the exams that it planned to use AI. 'The Committee was never informed about the use of AI before the exam took place, so it could not have considered, much less endorsed, its use,' Chan said. Katie Moran, an associate professor at the University of San Francisco School of Law who specializes in bar exam preparation, said this begged a series of questions. 'Who at the State Bar directed ACS Ventures, a psychometric company with no background in writing bar exam questions, to author multiple-choice questions that would appear on the bar exam?' she said on LinkedIn. 'What guidelines, if any, did the State Bar provide?' Mary Basick, assistant dean of academic skills at UC Irvine Law School, said it was a big deal that the changes in how the State Bar drafted its questions were not approved by the Committee of Bar Examiners or the California Supreme Court. 'What they approved was a multiple-choice exam with Kaplan-drafted questions,' she said. 'Kaplan is a bar prep company, so of course, has knowledge about the legal concepts being tested, the bar exam itself, how the questions should be structured. So the thinking was that it wouldn't be a big change.' Read more: 'It's a shambles': DOGE cuts bring chaos, long waits at Social Security for seniors Any major change that could impact how test-takers prepare for the exam, she noted, requires a two-year notice under California's Business and Professions Code. 'Typically, these types of questions take years to develop to make sure they're valid and reliable and there's multiple steps of review,' Basick said. 'There was simply not enough time to do that.' Basick and other professors have also raised concerns that hiring a non-legally trained psychometrist to develop questions with AI, as well as determine whether the questions are valid and reliable, represents a conflict of interest. The State Bar has disputed that idea: 'The process to validate questions and test for reliability is not a subjective one, and the statistical parameters used by the psychometrician remain the same regardless of the source of the question,' it said in a statement. On Tuesday, the State Bar told The Times that all questions were reviewed by content validation panels and subject matter experts ahead of the exam for factors including legal accuracy, minimum competence and potential bias. When measured for reliability, the State Bar said, the combined scored multiple-choice questions from all sources — including AI — performed 'above the psychometric target of 0.80.' The State Bar has yet to answer questions about why it deviated from its plan for Kaplan to draft all the exam multiple-choice questions. It has also not elaborated on how ACS Ventures used AI to develop its questions. Sign up for Essential California for news, features and recommendations from the L.A. Times and beyond in your inbox six days a week. This story originally appeared in Los Angeles Times.
