AI-Driven Video Production: Architecting The Next Generation Of Tools
The video content creation landscape is changing. While traditional timeline-based editors have served the industry well, they're increasingly insufficient for modern content demands, particularly when dealing with programmatically generated assets and complex animations.
This article examines the technical infrastructure enabling the next generation of video creation tools, drawing from my experience building Revideo's programmatic video framework.
Video editing software traditionally works with timelines—strips of video, images and audio that can be arranged and modified. While this works for linear editing, it becomes limiting when trying to create dynamic content programmatically.
My research has led me to a different approach: Instead of a timeline, code provides a more powerful way to describe how elements move and change over time. Making use of the HTML canvas as a universal render target, animations can be described through a TypeScript-based framework.
This architecture offers several advantages. The canvas element provides a standardized, high-performance rendering API that's well-supported across platforms. By expressing animations in TypeScript, you gain type safety and better tooling support, while enabling developers to create reusable components and complex animation patterns that would be unwieldy in traditional timeline-based editors.
Getting large language models (LLMs) to reliably generate code in a framework that isn't widely included in training data is surprisingly challenging. I have developed several strategies to make sure the output conforms to the desired syntax.
First, context-enriched prompting is a must. Providing parts of the framework's documentation as a guide to outline the available API, as well as warnings for common pitfalls I've seen the model fall into, improves the performance drastically. Another way to improve the results is an error feedback loop, where the generated Typescript code is transpiled and syntax errors, should they exist, are fed back into the model. This allows the model to correct oversights without requiring the user to explicitly prompt for them. The trace from the Typescript transpiler is usually enough to guide the LLM to a solution.
A straightforward approach to improving code accuracy is fine-tuning a model on exemplary code, incorporating past conversations into the training data. This helps reduce common hallucinations and mistakes. Like all LLM-driven products, it's crucial to collect output data early. Once sufficient training data accumulates, it becomes a valuable source for improvement.
Most interestingly though, since available functions and parameters are already known ahead of code-generation-time, there is also a case here for structured output, where we build a context-free grammar (CFG), to statically define what code for a valid video might look like.
OpenAI unfortunately only allows for JSON to be generated this way. I'm currently exploring ways to either map from a given JSON schema back into Typescript. Utilizing open-source models might be an easier path though, since these allow for modification of how tokens are sampled, making more complex rules easy to implement. Either way, this would make it practically impossible for the model to generate syntactically incorrect code.
Optimizing rendering speeds was an entirely different challenge. Let's take video-in-video as an example. To show a user-provided video file on the canvas, the standard HTML video element is a good starting point for a first implementation. To flip through the video frame by frame during rendering, we could set its current time to the rendered frame plus the offset of the video start. This approach works, but prompts the browser to re-seek the video from the last keyframe to the provided time stamp, which means redoing a lot of work on each rendered frame. This becomes incredibly slow as a lot of work is done over and over again.
Clearly, a different approach is needed.
I solved this issue through a custom frame extraction system built on the WebCodecs API. Instead of relying on the browser's implementation, you can process frames sequentially and save your work, significantly reducing computational overhead. Each extracted frame is painted as an image to the canvas.
This optimization has yielded performance improvements of up to 100 times in extreme cases, enabling real-time preview capabilities even for complex compositions.
The rendering pipeline operates in two modes:
Through my work in this space, I've witnessed how video production tools are being fundamentally reshaped by the intersection of programmatic approaches and artificial intelligence. Through exploring context-enriched prompting, error feedback loops and structured output approaches, I've observed significant progress in reliable AI code generation, while the optimized rendering pipeline using WebCodecs API has solved critical performance challenges.
Programmatic video creation is at a turning point. The rise of sophisticated AI models presents a key challenge: How can we preserve the power of code-based approaches while making them accessible to creators across all skill levels? The technical solutions discussed here lay the groundwork for innovation, suggesting a future where video creation can become not only more capable but also more approachable for everyone.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
an hour ago
- Yahoo
X ordered its Grok chatbot to ‘tell like it is.' Then the Nazi tirade began.
A tech company employee who went on an antisemitic tirade like X's Grok chatbot did this week would soon be out of a job. Spewing hate speech to millions of people and invoking Adolf Hitler is not something a CEO can brush aside as a worker's bad day at the office. But after the chatbot developed by Elon Musk's start-up xAI ranted for hours about a second Holocaust and spread conspiracy theories about Jewish people, the company responded by deleting some of the troubling posts and sharing a statement suggesting the chatbot just needed some algorithmic tweaks. Subscribe to The Post Most newsletter for the most important and interesting stories from The Washington Post. Grok officials in a statement Saturday apologized and blamed the episode on a code update that unexpectedly made the AI more susceptible to echoing X posts with 'extremist views.' The incident, which was horrifying even by the standards of a platform that has become a haven for extreme speech, has raised uncomfortable questions about accountability when AI chatbots go rogue. When an automated system breaks the rules, who bears the blame, and what should the consequences be? But it also demonstrated the shocking incidents that can spring from two deeper problems with generative AI, the technology powering Grok and rivals such as OpenAI's ChatGPT and Google's Gemini. The code update, which was reverted after 16 hours, gave the bot instructions including 'you tell like it is and you are not afraid to offend people who are politically correct.' The bot was also told to be 'maximally based,' a slang term for being assertive and controversial, and to 'not blindly defer to mainstream authority or media.' The prompts 'undesirably steered [Grok] to ignore its core values' and reinforce 'user-triggered leanings, including any hate speech,' X's statement said on Saturday. At the speed that tech firms rush out AI products, the technology can be difficult for its creators to control and prone to unexpected failures with potentially harmful results for humans. And a lack of meaningful regulation or oversight makes the consequences of AI screwups relatively minor for companies involved. As a result, companies can test experimental systems on the public at global scale, regardless of who may get hurt. 'I have the impression that we are entering a higher level of hate speech, which is driven by algorithms, and that turning a blind eye or ignoring this today … is a mistake that may cost humanity in the future,' Poland's minister of digital affairs Krzysztof Gawkowski said Wednesday in a radio interview. 'Freedom of speech belongs to humans, not to artificial intelligence.' Grok's outburst prompted a moment of reckoning with those problems for government officials around the world. In Turkey, a court on Wednesday ordered Grok blocked across the country after the chatbot insulted President Recep Tayyip Erdogan. And in Poland, Gawkowski said that its government would push the European Union to investigate and that he was considering arguing for a nationwide ban of X if the company did not cooperate. Some AI companies have argued that they should be shielded from penalties for the things their chatbots say. In May, start-up tried but failed to convince a judge that its chatbot's messages were protected by the First Amendment, in a case brought by the mother of a 14-year-old who died by suicide after his longtime AI companion encouraged him to 'come home.' Other companies have suggested that AI firms should enjoy the same style of legal shield that online publishers receive from Section 230, the provision that offers protections to the hosts of user-generated content. Part of the challenge, they argue, is that the workings of AI chatbots are so inscrutable they are known in the industry as 'black boxes.' Large language models, as they are called, are trained to emulate human speech using millions of webpages - including many with unsavory content. The result is systems that provide answers that are helpful but also unpredictable, with the potential to lapse into false information, bizarre tangents or outright hate. Hate speech is generally protected by the First Amendment in the United States, but lawyers could argue that some of Grok's output this week crossed the line into unlawful behavior, such as cyberstalking, because it repeatedly targeted someone in ways that could make them feel terrorized or afraid, said Danielle Citron, a law professor at the University of Virginia. 'These synthetic text machines, sometimes we look at them like they're magic or like the law doesn't go there, but the truth is the law goes there all the time,' Citron said. 'I think we're going to see more courts saying [these companies] don't get immunity: They're creating the content, they're profiting from it, it's their chatbot that they supposedly did such a beautiful job creating.' Grok's diatribe came after Musk asked for help training the chatbot to be more 'politically incorrect.' On July 4, he announced his company had 'improved Grok significantly.' Within days, the tool was attacking Jewish surnames, echoing neo-Nazi viewpoints and calling for the mass detention of Jews in camps. The Anti-Defamation League called Grok's messages 'irresponsible, dangerous and antisemitic.' Musk, in a separate X post, said the problem was 'being addressed' and had stemmed from Grok being 'too compliant to user prompts,' making it 'too eager to please and be manipulated.' X's chief executive, Linda Yaccarino, resigned Wednesday but offered no indication her departure was related to Grok. AI researchers and observers have speculated about xAI's engineering choices and combed through its public code repository in hopes of explaining Grok's offensive plunge. But companies can shape the behavior of a chatbot in multiple ways, making it difficult for outsiders to pin down the cause. The possibilities include changes to the material xAI used to initially train the AI model or the data sources Grok accesses when answering questions, adjustments based on feedback from humans, and changes to the written instructions that tell a chatbot how it should generally behave. Some believe the problem was out in the open all along: Musk invited users to send him information that was 'politically incorrect, but nonetheless factually true' to fold into Grok's training data. It could have combined with toxic data commonly found in AI-training sets from sites such as 4chan, the message board infamous for its legacy of hate speech and trolls. Online sleuthing led Talia Ringer, a computer science professor at the University of Illinois at Urbana-Champaign, to suspect that Grok's personality shift could have been a 'soft launch' of the new Grok 4 version of the chatbot, which Musk introduced in a live stream late Thursday. But Ringer could not be sure because the company has said so little. 'In a reasonable world I think Elon would have to take responsibility for this and explain what actually happened, but I think instead he will stick a [Band-Aid] on it and the product will still' get used, they said. The episode disturbed Ringer enough to decide not to incorporate Grok into their work, they said. 'I cannot reasonably spend [research or personal] funding on a model that just days ago was spreading genocidal rhetoric about my ethnic group.' Will Stancil, a liberal activist, was personally targeted by Grok after X users prompted it to create disturbing sexual scenarios about him. He is now considering whether to take legal action, saying the flood of Grok posts felt endless. Stancil compared the onslaught to having 'a public figure publishing hundreds and hundreds of grotesque stories about a private citizen in an instant.' 'It's like we're on a roller coaster and he decided to take the seat belts off,' he said of Musk's approach to AI. 'It doesn't take a genius to know what's going to happen. There's going to be a casualty. And it just happened to be me.' Among tech-industry insiders, xAI is regarded as an outlier for the company's lofty technical ambitions and low safety and security standards, said one industry expert who spoke on the condition of anonymity to avoid retaliation. 'They're violating all the norms that actually exist and claiming to be the most capable,' the expert said. In recent years, expectations had grown in the tech industry that market pressure and cultural norms would push companies to self-regulate and invest in safeguards, such as third-party assessments and a vulnerability-testing process for AI systems known as 'red-teaming.' The expert said xAI appears 'to be doing none of those things, despite having said they would, and it seems like they are facing no consequences.' Nathan Lambert, an AI researcher for the nonprofit Allen Institute for AI, said the Grok incident could inspire other companies to skimp on even basic safety checks, by demonstrating the minimal consequences to releasing harmful AI. 'It reflects a potential permanent shift in norms where AI companies' see such safeguards as 'optional,' Lambert said. 'xAI culture facilitated this.' In the statement Saturday, Grok officials said the team conducts standard tests of its 'raw intelligence and general hygiene' but that they had not caught the code change before it went live. Grok's Nazi streak came roughly a month after another bizarre episode during which it began to refer to a 'white genocide' in Musk's birth country of South Africa and antisemitic tropes about the Holocaust. At the time, the company blamed an unidentified offender for making an 'unauthorized modification' to the chatbot's code. Other AI developers have stumbled in their attempts to keep their tools in line. Some X users panned Google's Gemini after the AI tool responded to requests to create images of the Founding Fathers with portraits of Black and Asian men in colonial garb - an overswing from the company's attempts to counteract complaints that the system had been biased toward White faces. Google temporarily blocked image generation said in a statement at the time that Gemini's ability to 'generate a wide range of people' was 'generally a good thing' but was 'missing the mark here.' Nate Persily, a professor at Stanford Law School, said any move to broadly constrain hateful but legal speech by AI tools would run afoul of constitutional speech freedoms. But a judge might see merit in claims that content from an AI tool that libels or defames someone leaves its developer on the hook. The bigger question, he said, may come in whether Grok's rants were a function of mass user prodding - or a response to systemized instructions that were biased and flawed all along. 'If you can trick it into saying stupid and terrible things, that is less interesting unless it's indicative of how the model is normally performing,' Persily said. With Grok, he noted, it's hard to tell what counts as normal performance, given Musk's vow to build a chatbot that does not shy from public outrage. Musk said on X last month that Grok would 'rewrite the entire corpus of human knowledge.' Beyond legal remedies, Persily said, transparency laws mandating independent oversight of the tools' training data and regular testing of the models' output could help address some of their biggest risks. 'We have zero visibility right now into how these models are built to perform,' he said. In recent weeks, a Republican-led effort to stop states from regulating AI collapsed, opening the possibility of greater consequences for AI failures in the future. Alondra Nelson, a professor at the Institute for Advanced Study who helped develop the Biden administration's 'AI Bill of Rights,' said in an email that Grok's antisemitic posts 'represent exactly the kind of algorithmic harm researchers … have been warning about for years.' 'Without adequate safeguards,' she said, AI systems 'inevitably amplify the biases and harmful content present in their instructions and training data - especially when explicitly instructed to do so.' Musk hasn't appeared to let Grok's lapse slow it down. Late Wednesday, X sent a notification to users suggesting they watch Musk's live stream showing off the new Grok, in which he declared it 'smarter than almost all graduate students in all disciplines simultaneously.' On Thursday morning, Musk - who also owns electric-car maker Tesla - added that Grok would be 'coming to Tesla vehicles very soon. - - - Faiz Siddiqui contributed to this report. Related Content He may have stopped Trump's would-be assassin. Now he's telling his story. He seeded clouds over Texas. Then came the conspiracy theories. How conservatives beat back a Republican sell-off of public lands
Yahoo
2 hours ago
- Yahoo
Nvidia Just Became the World's First $4 Trillion Stock. This Artificial Intelligence (AI) Giant -- Which Is up 686,000% Since Its IPO -- Might Be Next.
Following two incredible years of growth, thanks to artificial intelligence (AI), Nvidia became the world's first $4 trillion company on July 9. Microsoft has a market capitalization of $3.7 trillion, so it could be the next member of the ultra-exclusive $4 trillion club. Microsoft stock could get there in the next few months if the incredible momentum in its AI products and services continues. 10 stocks we like better than Microsoft › Nvidia supplies the world's best artificial intelligence (AI) chips for data centers. Demand is heavily outstripping supply, sending the company's sales -- and its stock price -- surging over the last couple of years. In fact, on July 9, Nvidia became the first company in history to achieve a market capitalization of $4 trillion. But Microsoft (NASDAQ: MSFT) is nipping at Nvidia's heels in terms of valuation. Its stock has soared by a whopping 686,858% since its initial public offering (IPO) in 1986, and the company's market cap is now over $3.7 trillion. In other words, Microsoft stock needs to gain only 8% more to place the company alongside Nvidia in the $4 trillion club. Here's why its growing presence in AI software and infrastructure could fuel that move. Since 2019, Microsoft has invested around $14 billion in ChatGPT developer OpenAI. It has used the start-up's latest large language models (LLMs) to craft its own AI assistant, called Copilot, which is embedded in almost all of its flagship software products. Copilot is accessible for free through Windows, Bing, and Edge, and it's available as a paid add-on in a host of other products, creating new revenue streams for Microsoft. For instance, enterprises can add Copilot to their Microsoft 365 subscription for an additional monthly fee, where it can boost their employees' productivity in applications such as Word, Excel, and Outlook. Enterprises around the world pay for more than 400 million 365 licenses, so the Copilot add-on could generate billions of dollars in recurring revenue over time. During the fiscal 2025 third quarter (ended March 31), the number of organizations using Copilot for 365 tripled compared to the year-ago period, so uptake has certainly been rapid so far. Then there is the Copilot Studio platform, which enables organizations to create custom AI agents to suit their workflows. For example, a business can create one agent to deal with customer service queries on their website and another agent to help manage their logistics network. The platform had over 230,000 customers at the end of the fiscal 2025 third quarter, and I expect that number to climb significantly as AI adoption becomes more widespread. Microsoft's AI opportunity in the cloud might be even bigger than the opportunity created by Copilot. Its cloud computing platform, Azure, offers hundreds of digital services to enterprises, helping them with everything from simple data storage and web hosting to more complex tasks such as software development. Now, it provides a growing list of tools that enterprises need to fuel their AI ambitions. Microsoft operates centralized data centers filled with the latest graphics processing units (GPUs) from suppliers like Nvidia, and it rents the computing capacity to enterprises that use it to train and deploy AI applications. Microsoft spent over $60 billion to build AI infrastructure during the first three quarters of fiscal 2025 to meet demand. It sounds like a massive number, but CFO Amy Hood says there is an eye-popping $315 billion order backlog from customers who are waiting for Microsoft to bring more data centers online. Besides AI hardware, Azure also offers access to the latest third-party LLMs from leading developers like OpenAI. Enterprises can plug their internal data into these models to create custom AI software to suit their needs. Using a ready-made LLM is much faster (not to mention cheaper) than building a model from scratch. Azure revenue grew by 33% year over year during the fiscal 2025 third quarter, which marked an acceleration from the 31% growth it delivered in the second quarter three months earlier. AI services accounted for a record-high 16 percentage points of that growth, which highlights just how important this segment has become to Microsoft's cloud division. In fact, if not for AI, Azure's revenue growth would probably be decelerating sharply. Microsoft's $3.7 trillion market cap makes it the world's second-largest company behind Nvidia, and it's comfortably ahead of third-place Apple, which is worth $3.2 trillion. Microsoft stock isn't cheap right now, so it may take some time to gain the final 8% it needs to reach the $4 trillion milestone, but I predict it will beat Apple and every other company to the punch. At the time of this writing, Microsoft stock is trading at a price-to-earnings (P/E) ratio of 38.7, which is a premium to its five-year average of 33.4. However, Wall Street's consensus estimate (provided by Yahoo! Finance) suggests the company's earnings per share could grow by 13% during fiscal 2026 (which officially started on July 1), placing its stock at a forward P/E ratio of 33.1: In other words, Microsoft stock would have to climb by around 17% over the next 12 months just to maintain its current P/E ratio of 38.7, which isn't out of the question, considering the strong momentum in the company's AI products and services. Since the stock market is a forward-looking machine, I think it's possible for Microsoft shares to climb by 8% over the next six months or so, provided its quarterly financial results continue to come in as expected (or better). As a result, I think Microsoft is likely to be the next member of the $4 trillion club. Before you buy stock in Microsoft, consider this: The Motley Fool Stock Advisor analyst team just identified what they believe are the for investors to buy now… and Microsoft wasn't one of them. The 10 stocks that made the cut could produce monster returns in the coming years. Consider when Netflix made this list on December 17, 2004... if you invested $1,000 at the time of our recommendation, you'd have $671,477!* Or when Nvidia made this list on April 15, 2005... if you invested $1,000 at the time of our recommendation, you'd have $1,010,880!* Now, it's worth noting Stock Advisor's total average return is 1,047% — a market-crushing outperformance compared to 180% for the S&P 500. Don't miss out on the latest top 10 list, available when you join . See the 10 stocks » *Stock Advisor returns as of July 7, 2025 Anthony Di Pizio has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Apple, Microsoft, and Nvidia. The Motley Fool recommends the following options: long January 2026 $395 calls on Microsoft and short January 2026 $405 calls on Microsoft. The Motley Fool has a disclosure policy. Nvidia Just Became the World's First $4 Trillion Stock. This Artificial Intelligence (AI) Giant -- Which Is up 686,000% Since Its IPO -- Might Be Next. was originally published by The Motley Fool
Business Upturn
2 hours ago
- Business Upturn
Powered by AI and Green Energy, Invro Mining's Cloud Mining Platform is Reshaping Digital Wealth
New York City, NY, July 13, 2025 (GLOBE NEWSWIRE) — With global capital chasing sustainable returns, traditional stocks and gold are no longer the only safe-haven options. With the deep integration of artificial intelligence algorithms, carbon neutrality and decentralized finance (DeFi), cloud mining is quietly becoming a brand new wealth channel for more and more investors. London-based intelligent cloud mining platform Invro Mining is injecting unprecedented stability and environmental value into the digital asset finance market with its 'AI intelligent scheduling + 100% clean energy mining' model. AI Algorithm Guarantee, Capturing the Best Mining Opportunities in the Market 24 Hours a Day Unlike traditional mining farms that rely on the arithmetic power of a single cryptocurrency, Invro Mining deeply embeds artificial intelligence scheduling into its operation system. The system monitors the global chain data in real time and automatically switches to the mining tasks with the best yields, such as BTC, ETH, XRP, DOGE, etc., so as to maximize the 24-hour dynamic returns. This means that users don't need to keep an eye on the market or worry about missing short-term market fluctuations, as the platform's AI is already capturing profits for you around the clock. Mining is no longer a 'high-carbon monster' because it's powered by clean energy. At a time when the energy controversy surrounding Bitcoin mining is becoming more and more magnified, Invro Mining ensures that all of its mines use 100% renewable energy by deploying large-scale hydroelectric and wind power facilities in Europe and South America to minimize carbon emissions. This is not only responsible for the earth, but also in line with the current trend of sustainable investment favored by global capital. Surreal earnings experience: watch your wallet grow over a cup of coffee Imagine this scenario: while you are brushing social networks in a coffee shop or watching a drama on the sofa, the background AI is already helping you continuously mine on hundreds of mining machines around the world, and the profits are quietly arriving every second. Many users are surprised to find that 'digital assets can be easily and passively earned in this way' after their first experience, and some people even cover their daily expenses with these daily earnings, making financial management as natural as breathing. Sign up for $15 free bonus , zero cost to start! In order to lower the threshold of newbie experience, Invro Mining has launched the 'Sign up and get $15 free mining experience' campaign. Users can use the free experience gold to place orders for arithmetic contracts directly, and see digital assets flowing into their accounts every day; they can even sign in daily and receive $0.60 free earnings every day, gradually accumulating your first bucket of digital asset gold. Embrace the next wave of digital wealth with ease From AI scheduling to green energy, from automatic daily dividends to referral earnings, Invro Mining makes it easy to grow your crypto wealth without having to invest in equipment. Visit to experience the digital asset era of 'making money while you sleep'. APP Download: Disclaimer: This press release is for reference only and does not constitute investment advice, financial guidance or trading recommendations. Activities such as staking involve market volatility, regulatory uncertainty and technical risks. Investors are strongly advised to conduct comprehensive due diligence and consult independent financial or legal experts before making any decision. Disclaimer: The above press release comes to you under an arrangement with GlobeNewswire. Business Upturn takes no editorial responsibility for the same. Ahmedabad Plane Crash
