ByteDance says new AI technology boosts model training efficiency by 1.7 times
TikTok owner ByteDance said it has achieved a 1.71 times efficiency improvement in large language model (LLM) training, the latest Chinese tech company to achieve a breakthrough that could potentially reduce demand for Nvidia's high-end graphics processing units (GPUs).
Advertisement
The company's Doubao development team said they managed to 'speed up' LLM training efficiency by '1.71 times' through COMET, an optimised Mixture-of-Experts (MoE) system, according to a recent paper published on arXiv, an online forum for professionals in the scientific community.
MoE is a machine learning technique where multiple expert networks are used to divide a problem space into homogeneous sections. The technique has been extensively adopted to scale LLMs to trillion-plus parameters, while maintaining fixed computing cost. It is widely adopted by leading artificial intelligence (AI) models such as Grok and DeepSeek.
The headquarters of ByteDance is seen in Beijing on September 16, 2020. Photo: AFP
The new system has already been adopted in the company's production environment of clusters using over 10,000 GPUs, achieving 'savings of millions of GPU hours', according to the Doubao team.
Breakthroughs in reducing the training cost for AI models could lead to lower demand for chips from Nvidia, whose high-performance GPUs are subject to strict export controls by the US.
The rise of Hangzhou-based DeepSeek, which developed and trained its AI models at a fraction of the cost and with fewer computing resources than its Western counterparts, had led to speculation that demand for Nvidia GPUs could slow. After the DeepSeek breakthrough, Nvidia experienced a nearly US$600 billion drop in market value last month, in the largest single-day drop for any US company, before it rebounded the next day.
Advertisement
Although widely adopted by major tech companies, the MoE technique leads to so-called 'communication-computation overlapping', which 'introduces a notable impairment of computational efficiency', according to the ByteDance scientists. The new system is aimed at addressing this problem by 'eliminating fine-grained communication bottlenecks and enhancing its adaptability across various scenarios', they wrote.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Asia Times
6 hours ago
- Asia Times
America and rare earths: a little urgency, please
America usually has other countries over a barrel. Not the other way around – unless you're old enough to remember when the Organization of Petroleum Exporting Countries (OPEC) cracked the whip and Washington and the West usually fell in line. But last week China forced President Trump to back down on tough tariffs that were hurting the PRC. Why the retreat? Rare earth minerals. They're nearly all sourced from China and Beijing choked off exports. Major American companies – car makers for example – warned Trump they'd have to curtail or shut down operations in a matter of weeks. That's never good for an incumbent party's political prospects. The Trump administration reduced tariffs in exchange for Beijing's sort-of agreement to allow rare earth exports to US customers. How much and how fast is unknown – but expect the PRC to slow-roll this and squeeze all it can – such as sensitive business data – from its customers. At best, this is a temporary reprieve. The US side also agreed to drop plans to ban Chinese students from US universities. One also suspects a quiet agreement to relax US controls on semiconductor and AI-related technology exports to China. US tariffs remain at 55%, but that's much lower than the 145% originally imposed. And the PRC probably can handle that level. There's still a 20% 'fentanyl tariff' imposed for Chinese-origin fentanyl that has killed well over 500,000 Americans over the last decade. This ought to be 2000%. Instead, US negotiators settled for China agreeing (once again) to discuss the fentanyl issue. China controls around 90% of the world's rare earth processing. And America imports most of its rare earths from the PRC. It's not that they are rare. The US has plenty of rare earths. It's just much cheaper to source from China, and US environmental laws make domestic mining and processing difficult and expensive. They are essential for commercial manufacturing – cars, electronics, computers, etc. And more ominously they're needed for military production – to include aircraft, ships, submarines, radars, missiles, lasers, satellites, guidance systems, night vision devices and more. It's not Trump's fault. He got caught holding the hot potato. China's chokehold on rare earths was known a long time ago. American business and the US military knew. And Congress knew as well. And they knew the Chinese might use their dominant position to squeeze other countries. In 2010 after Japan detained a Chinese fishing boat that rammed a Coast Guard ship near Japan's Senkaku islands, China banned rare earth exports to Japan. Japanese industry squirmed. The boat and its skipper were returned. And Japan set about finding alternate sources of rare earths. The United States? Apparently it did nothing. It was 15 years of idiocy by the business class, officialdom and the military's perfumed princes. Yes, some people sounded the alarm, but they got nowhere. Being known as 'the rare earths congressman' wasn't a vote-getter. The Obama and Biden administrations' environmental zealots were glad to have 'dirty' rare earths produced elsewhere – but not in America. Businessmen are all about efficiency, low cost and maximizing profit. Rare earths would always be available somewhere – and anyway the Chinese wouldn't choke the flow. After all, the Chinese weren't an enemy. They just wanted to get rich. Such rationalizations come easy to the MBA class. But rare earth vulnerability was also not a priority for America's military leaders. Think about this. Nobody saw anything wrong with being dependent – over 90% dependent – on the PRC for a crucial resource that the military needs to fight? The PRC had declared America 'the main enemy' and has built a military intended for one main purpose: to defeat the US military. But they would keep supplying the US military with vital materials – without which it can barely function? Who in the top ranks was raising merry hell about this state of affairs? Nobody. Instead, it was Legions of Merit and promotions all around. The rare earth problem is now getting attention. It has been for a few years. The Trump administration has bought a little time, but the PRC will use rare earths as a Sword of Damocles as long as possible. Still there's no sense of urgency. 'Drill baby drill' and Yankee ingenuity got American energy independence back in Trump 1.o – even if Team Biden later decided it didn't like the idea. So maybe it will be 'Rare earths baby rare earths' this time around. If so, restoring tax credits for domestic critical minerals mining and processing to the administration's 'Big Beautiful Bill' is essential. Pundits say it will take ten years to break the China dependency. They are often wrong. When Americans put their minds to something – and are given freedom to operate – things always happen faster. But first stop admiring the problem and get moving. Faster. Grant Newsham is a retired US Marine officer and former US diplomat. He's a fellow at the Center for Security Policy and the Yorktown Institute, and is the author of When China Attacks: A Warning to America . Follow on X @NewshamGrant
Asia Times
6 hours ago
- Asia Times
China's fast-growing high-speed railway network faces reality
The financial situation of China's high-speed railway network has raised concerns among Chinese commentators as the facility is facing growing debt and weakening passenger spending power. In February this year, a group of Chinese commentators said a report by the National Audit Office (NAO) had found that China's high-speed railway saw an 'about 100 billion yuan of total loss' in the nine months ending December 31, 2024. NAO's website did not officially announce this widely reported figure. Since then, Chinese commentators have started debating the matter. Some said the country should not have extended its high-speed railway to remote places without calculating the costs over the past two decades. Others said China's high-speed railway has value as a public facility that connects people in less developed areas to large cities. Specific financial figures about China's high-speed railway are unavailable, as the China State Railway Group Co Ltd (China Railway) combined them with those of 'green-skinned trains,' which are the older type of passenger trains often painted green. Although Beijing-Shanghai High Speed Railway Co Ltd regularly discloses its financial data to the Shanghai stock exchange, its figures do not reflect the situation of the loss-making units under China Railway. An article by said on June 5 that the Beijing-Shanghai High Speed Railway saw a 2.31% year-on-year decline in its total passenger number to 52.02 million people last year. The company still managed to boost its net profit by 10.6% to 12.77 billion yuan (US$1.78 billion) in 2024 by sharing its network with other train operators. The article pointed out that China's high-speed railway network was 45,000 kilometers at the end of 2023, but only 2,300 kilometers, or 6% of the total, could make a profit. It said that out of all 16 high-speed railway lines, only six in coastal cities are profitable. They include the Beijing-Shanghai, Beijing-Tianjin, Shanghai-Hangzhou, Ningbo-Hangzhou, and Guangzhou-Shenzhen lines. It said the most profitable Beijing-Shanghai line will have to spend 20 years recovering its initial investment of 220.9 billion yuan. Last year, China's train passengers surged 10.9% year-on-year to 4.09 billion in 2024, according to the National Railway Administration. China Railway's total revenue grew 3.1% to 1.28 trillion yuan, while net profit surged 17.6% to 3.88 billion yuan. These figures, which included the company's old railway and high-speed railway segments, could not reflect the situation in which many people had become price-sensitive in a sluggish Chinese economy. 'Since the beginning of 2024, data from many high-speed rail lines have been unsatisfactory,' a Henan-based writer says in an article published in February. 'There were very few passengers on weekdays, but the maintenance costs stood high.' 'In June 2024, the situation worsened, increasing financial pressure on the national railway operator,' he writes. 'Last September, the government finally took action by urging the train operator to reschedule its trains' timetables, attract private investment and improve efficiency with new technologies.' He says that after lowering train frequency and fares in remote areas in January this year, the situation slightly improved, but not enough to boost China Railway's profitability. Media reports said that when millions of migrant workers went home during the Spring Festival in late January, many took the 'green-skinned trains,' leaving the high-speed railway trains underutilized. A Sichuan-based columnist says it takes about 4.5 hours to travel from Zhengzhou to Wuhan on ordinary trains for 70 to 90 yuan. He says the journey can be less than two hours on high-speed trains, but the fare is about 270 yuan. He says it's normal for migrant workers to take a slower train as the faster one would cost them one to two days' salary. On June 15, China Railway increased fares by up to 20% for its profitable lines to subsidize the unprofitable ones. According to its annual report, China Railway had total liabilities of 6.2 trillion yuan at the end of 2024, up 1.2% from 6.13 trillion yuan a year earlier. Total assets increased 4.4% to 9.76 trillion yuan from 9.35 trillion yuan for the same period. Since 2010, China Railway has closed 20 high-speed railway stations due to insufficient passenger traffic. Many 'ghost stations' were in inland provinces, such as Anhui and Yunnan, while some were in coastal provinces, such as Liaoning and Jiangsu. A Hubei-based columnist says that after the central government launched a 4 trillion yuan economic stimulus package in 2008, many local governments got new funding to accelerate their high-speed railway projects, but built many stations in remote and inconvenient places. He says that these local governments are now facing rising financial pressure and have to reduce the subsidies for their transportation network. This is why many stations have to be shut down. A Hebei-based writer named Zelin says many high-speed railway projects in China are doomed to lose money, as many local governments only wanted to achie ve political achievements without considering the profitability of the ir facilities. He said the maximum fare of the Beijing-Shanghai line is 553 yuan, which is lower than that of the Tokyo-Osaka line (the equivalent of 1,200 yuan). Thus, it is unlikely that China's high-speed railway projects will turn a profit in the short run. He added that someone has suggested that China Railway increase train fares, encourage local governments to bear more construction costs, cut the number of high-speed railway stations from 1,300 to 960, improve governance, and reduce costs. However, he said that all these suggestions are easier said than done. He said the public should focus more on the high-speed railway network's social value than its commercial value. Read: Price war sparks EV financial crisis concerns in China
South China Morning Post
11 hours ago
- South China Morning Post
Huawei and ByteDance plan major investments in tech sectors in Brazil
Chinese tech giants Huawei Technologies and ByteDance are planning major investments in Brazil's cloud infrastructure and artificial intelligence sectors, moves likely to deepen US concerns over Beijing's expanding digital reach in Latin America. Advertisement The Brazilian newspaper Folha de S. Paulo reported on Monday that Huawei is set to announce a deal with Dataprev – a state-run technology company that manages the country's social data systems – to use its data centres. Huawei is also in talks with Edge UOL, a cloud services arm of Grupo UOL PagSeguro, which also owns Folha. Executives from Huawei and Edge UOL met in May in Dongguan, China, along with Brazil's secretary of digital governance Ricardo Leite and the Huawei Cloud division's Latin America president Mark Chen. 'We want to be the bridge between China and Latin America,' Chen said, calling the Brazilian firm a 'strategic service partner'. Edge UOL Chief Operating Officer Rodrigo Lobo said that the partnership aimed to expand into infrastructure, cybersecurity and AI operations across Brazil. Rodrigo Lobo of Edge UOL (left) with Mark Chen of Huawei's Cloud division. Photo: Handout The planned expansions come as the United States has stepped up warnings about Chinese investments in critical tech infrastructure across Latin America, citing risks of data theft, surveillance and strategic leverage.
