Latest news with #securedevelopment

Forbes

AI Security Demystified: Practical Insights For Nontechnical Leaders

​Alex Pinaev, CEO of Mobix, is a software engineering expert with Fortune 500 consulting experience in secure development. Machine learning (ML) and generative AI (GenAI) are reshaping the organizational landscape. Companies increasingly recognize that AI drives innovation, helps sustain competitiveness and boosts workforce productivity. It has also become clear that internal company data can provide a unique competitive advantage for AI-powered applications. However, as more developers build solutions on top of large language models (LLMs), organizations are learning that truly cutting-edge results often require complex AI systems, far beyond simple prompt-based tools or plugins. These multicomponent LLM applications open the door to new capabilities but also introduce serious risks. Companies are rightfully concerned about data loss, breaches of confidentiality, model theft and the challenges of meeting current and future compliance obligations when using proprietary data for ML and GenAI. Without strong access controls, users may unintentionally gain access to sensitive information. In customer-facing applications, this can even result in data leaks to competitors. The absence of auditability and traceability further increases the risk of noncompliance. This article is intended for practitioners who understand that GenAI must be implemented—not feared—and who are actively seeking tools and processes to reduce exposure and errors. The Expanding Threat Surface Of LLM Applications It is becoming increasingly obvious that LLMs significantly expand the attack surface. Even relatively simple prompt-based applications are exposed to security risks. These include prompt injection attacks and data leakage caused by poorly constructed or malicious prompts. The absence of context awareness or session management can result in unpredictable or unintended behavior. Additionally, user-generated prompts may produce inconsistent or biased outputs, raising concerns related to regulatory compliance and ethical standards. Agent-based LLM applications, which operate autonomously and interact with multiple systems, pose even greater risks. They are vulnerable to unauthorized access, and their integration across APIs widens the attack surface further. Moreover, weaknesses in their decision making logic can be exploited by attackers. If attackers compromise an autonomous agent, the consequences may be critical, particularly in healthcare, finance or infrastructure. To mitigate such risks, strong access controls, end-to-end encryption and secure APIs are essential. Data integrity and confidentiality must be enforced at rest, in transit and at the interface level. Complex LLM applications face additional challenges due to tight integration with enterprise systems and the handling of large volumes of sensitive data. Common issues include API vulnerabilities, adversarial input attacks and misconfigurations that can result in unauthorized access. The complexity of such systems also makes it harder to maintain compliance across all components. LLM Security Tools: What's Emerging The development of LLM-based systems is still in its early stages, and their risks often fall outside the scope of traditional cybersecurity or DevSecOps practices. While existing standards help reduce risks in conventional software systems, LLMs pose new, model-specific challenges. These models may be trained on data that changes over time, impacting reliability and behavior in ways that are hard to explain. LLMs also operate in complex, socio-technical environments influenced by human behavior, policy and social norms, making failure modes difficult to detect. Static code analysis won't prevent prompt injection or reveal ethical and social biases in model outputs. Understanding the open-source components of an AI agent is useful but far from sufficient. Defending LLM-based systems requires a fresh security strategy and a new development-testing-deployment lifecycle. The good news is that we are beginning to see emerging solutions aimed at closing these security gaps. One of the most promising developments is the introduction of LLM firewalls: security layers specifically designed to protect large language models from unauthorized access, malicious queries and potentially harmful outputs. These firewalls monitor and filter interactions with LLMs, blocking suspicious or adversarial inputs that could influence the model's behavior. They also enforce predefined rules and policies to ensure that the model only responds to legitimate queries within clearly defined ethical and operational boundaries. In addition, LLM firewalls help prevent data leakage and safeguard confidential information by controlling the flow of data into and out of the model. Another emerging category includes LLM vulnerability scanners: specialized tools developed to identify and assess risks unique to large language models. These scanners can detect prompt injection attempts and adversarial input attacks. They evaluate model behavior across various scenarios to uncover vulnerabilities that might go unnoticed by traditional security tools. Three Things You Should've Done Yesterday In time, we can expect the emergence of formal threat models, maturity assessment frameworks, certification standards and a new class of seasoned consultants specializing in LLM security. Until then, here's what pragmatic leaders should do now: Start by identifying whether the system is already in production and clarifying what stage of its lifecycle it's currently in. Document which internal or external systems the application interacts with, and what LLM engine powers it. Specify where the model's source code is stored and who is responsible for owning and maintaining the application. Review the available documentation and note whether a specific training framework was used. Determine whether the system is capable of making autonomous decisions, and finally, assess whether there are any safeguards in place to secure the interaction channel between clients and the model. Stay informed. Security in the GenAI era requires continuous learning. If you have the courage to experiment in this entirely new field on your own, I'd recommend the following hands-on exercise: Deploy any LLM, install a vulnerability scanner designed for language models and observe firsthand how obfuscation and prompt injection attacks work in practice. You might even succeed in making the model produce intentionally false conclusions or reveal secrets it was never meant to disclose. You'll quickly see that LLM vulnerabilities are unlike anything you've encountered in the world of traditional web application security. You'll need a budget, whether for safeguards, audits, tooling or rearchitecture. Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?