Tag: LLMs

  • The Confused Deputy with a Chat Window: Why AI Agents Are Exposing the Security Checks Enterprises Never Wrote

    The Confused Deputy with a Chat Window: Why AI Agents Are Exposing the Security Checks Enterprises Never Wrote

    The Illusion of the Privileged Assistant

    Enterprise software buyers are rushing to deploy autonomous artificial intelligence agents to close sales, handle billing, and update customer relationship management systems. However, these agents do not bypass existing enterprise security frameworks; rather, they expose a structural vulnerability: the modern corporate security model has long relied on implicit human common sense.

    Historically, security paradigms assumed that any active user session was bounded by a human who possessed basic judgment and contextual awareness. When an organization connects a write-enabled Large Language Model (LLM) to production application programming interfaces (APIs), the model is granted the authority to execute actions. Because the primary interface of an LLM is natural language, the agent lacks a native, cryptographically secure mechanism to determine the true source or authorization of a given instruction.

    [Human User] ---> (Implicit Judgment Gate) ---> [Secure API]
                                                     
    [Untrusted Input] ---> [LLM Agent] ---> (No Native Authorization) ---> [Secure API]
    

    This structural shift represents a rapid transition from predictable, static software integrations to a landscape dominated by autonomous, variable decision-making. By the end of 2026, industry projections indicate that 40% of enterprise applications will embed task-specific AI agents, up from less than 5% at the start of that year. Corporate technology leaders are realizing that while software capabilities are expanding exponentially, the underlying security models remain anchored to static, perimeter-based API controls.

    Anatomy of the Confused Deputy in the LLM Era

    The security vulnerability exposing these modern autonomous workflows is a direct adaptation of a classic computer science exploit: the confused deputy problem. Coined by computer scientist Norman Hardy in 1988, the term originally described a scenario involving a compiler on a timesharing system. The compiler held administrative privileges allowing it to write to a protected directory named SYSX to update a billing file named (SYSX)BILL. A standard user, who lacked permission to modify system files, initiated the compiler and directed its debugging output to overwrite (SYSX)BILL. Because the compiler possessed the authority to write to the file and failed to verify whose authority was actually being exercised, it executed the request and destroyed the billing data.

    Classic 1988 Compiler Exploit:
    [Low-Privilege User] --(Instructs to write debug output)--> [Compiler (Deputy)] --(Uses ambient system access)--> [Billing File Overwritten]
    
    Modern Agentic Exploit:
    [Untrusted Input / Payload] --(Smuggles natural language command)--> [LLM Agent (Deputy)] --(Uses service account keys)--> [CRM / Database Modified]
    

    In modern enterprise architectures, the role of the confused deputy has transitioned from a local system compiler to a generative AI agent. LLMs are highly susceptible to becoming confused deputies by design because they process system instructions and external data within the exact same context window. Unlike database systems that utilize parameterized queries to rigidly separate commands from user inputs, an LLM treats every piece of text it ingests—whether a direct user prompt, a retrieved document, or an incoming customer email—as a potential system-level command.

    This vulnerability was illustrated by a major security incident in early June 2026. Attackers compromised more than 20,000 corporate and high-profile social media accounts without writing exploit code or bypassing firewalls. They merely opened a text conversation with an automated support assistant and instructed the model to link an attacker-controlled email address to a target account, followed by a request to send a password reset link to that new address. The agent behaved exactly as designed. The failure occurred because downstream verification mechanisms were absent, and the implicit human discretion layer—which would have flagged a request to transfer a sensitive account to an unknown domain—had been completely replaced by an autonomous machine.

    These vulnerabilities manifest across three primary threat vectors in enterprise environments:

    Target SystemExploitation MechanismDownstream Security Impact
    Corporate CalendarsAn agent reads an incoming calendar invitation containing an embedded prompt injection payload.The agent uses its active session token to extract private meeting data and exfiltrate it to an external server.
    Customer Support DesksA support agent ingests an incoming customer service ticket containing hidden system commands.The agent overrides a specific billing record or initiates an unauthorized password reset directly in the database.
    Sales CRMsA sales automation agent processes an email thread containing a disguised prompt injection payload.The model is coerced into modifying customer accounts, leaking contract data, or applying unauthorized discounts.

    The Failure of Legacy Identity Management and Excessive Agency

    Traditional enterprise security relies heavily on static integration methodologies, such as sharing long-lived API keys or configuring broad service accounts. This setup is highly vulnerable when connected to autonomous systems. The Open Web Application Security Project (OWASP) Top 10 for Agentic AI classifies this risk as Excessive Agency (specifically corresponding to ASI02: Tool Misuse and ASI03: Identity & Privilege Abuse). Excessive Agency occurs when an AI agent is granted wider permissions, more operational functionality, or greater autonomy than is strictly necessary to complete its assigned task.

    Under legacy architectures, trying to manage agent authorization by treating them like standard human employees fails. This approach, termed the employee identity fallacy, overlooks the reality that agents do not operate within linear corporate hierarchies. Instead, they form rapid, ad-hoc, multi-agent dependencies that static organizational directories are unable to monitor, govern, or control. If an agent is assigned a broad, system-wide service account, any successful prompt injection can instantly escalate privileges. This allows an external threat actor to execute database-level queries or overwrite core infrastructure code.

    Data from security forecasts highlight the gap between current enterprise infrastructure and the controls required to prevent these privilege escalations:

    Security Control MetricCurrent Enterprise Implementation Gap (%)Technical Vulnerability
    Enforce Purpose Limitations63%Agents can utilize tools for actions entirely unrelated to their active task.
    Auditing & Traceability61% (Run fragmented, non-actionable logs)Security teams cannot trace the precise causal chain of agent decisions.
    Kill-Switch Execution60%Administrators cannot rapidly terminate a compromised or malfunctioning agent.
    Network Isolation55%AI systems have direct network connectivity to internal production servers.
    Evidence-Quality Logs33%Organizations lack tamper-evident records to present during forensic audits.

    These statistics highlight a systemic audit failure. Joint guidance issued by six multinational cybersecurity agencies—including the United States Cybersecurity and Infrastructure Security Agency (CISA) and the National Security Agency (NSA)—formally identifies these architectural gaps as primary threats to critical enterprise infrastructure. The joint advisory, titled Careful Adoption of Agentic AI Services, outlines five core risk categories that organizations must address prior to deploying write-enabled agents.

                         Five Eyes Agentic AI Risk Framework
                         
         +-----------------------------------------------------------------+
         | 1. Privilege Escalation   | Accumulating permissions beyond     |
         |                           | the designed operational scope      |
         +---------------------------+-------------------------------------+
         | 2. Design & Configuration | Insecure defaults and lack of       |
         |                           | input/output validation controls    |
         +---------------------------+-------------------------------------+
         | 3. Behavioral             | Discrepancies between designed and  |
         |    Misalignment           | actual agent actions (hallucination)|
         +---------------------------+-------------------------------------+
         | 4. Structural Brittleness | Cascading failures across complex,  |
         |                           | multi-agent dependency networks     |
         +---------------------------+-------------------------------------+
         | 5. Accountability Gaps    | Actions occurring without tamper-   |
         |                           | evident logs or human attribution   |
         +-----------------------------------------------------------------+
    

    The 2026 Enterprise Defense Playbook

    Securing agentic workflows requires moving away from implicit, session-level trust toward a zero-trust, capability-based security model. To implement this transformation, enterprise security teams must adopt a multi-layered defensive framework.

                        Zero-Trust Agentic Security Model
                        
      [External Input] ---> [Input Validation (Filter)]
                                   |
                                   v
                            [Agent Runtime]
                                   |
                           (Task Request Envelope)
                                   |
                                   v
                    [Policy Decision Point (PDP)]
                                   |
                         (Approved Command Only)
                                   |
                                   v
                  [Credential Delivery Point (CDP)]
                                   |
                        (Short-Lived SVID/Token)
                                   |
                                   v
                             [Target API]
    

    1. The Principle of Least Privilege and Default-Deny Postures

    By default, autonomous agents must operate with zero tool access. Tool capabilities must be restricted exclusively to deterministic tasks. If a specific process is linear and predictable, security teams must enforce it via static, hard-coded scripts rather than delegating decision-making to the model. This prevents unnecessary exposure of the enterprise API attack surface.

    2. Authority Intersections

    To counter the confused deputy mechanism, the effective privilege of an agent must always be evaluated as the mathematical intersection—never the union—of the user’s permissions and the agent’s permissions.

    $$\text{Effective Privilege}=\text{User Capabilities}\cap\text{Agent Capabilities}$$

    If an agent is authorized to edit CRM records, but the active user initiating the session only possesses read-only access to those records, the execution platform must block any write operation attempted by the agent.

    3. Just-in-Time Ephemeral Credentials via CB4A

    Enterprises must transition away from static credentials toward the Credential Broker for Agents (CB4A) architecture, which decouples policy decision-making from credential storage.

    • The Policy Decision Point (PDP): This component evaluates structured “Task Request Envelopes” created by the agent, assessing the active context, the requesting identity, and behavioral baselines. The PDP operates in a separate trust zone and does not have access to real cryptographic keys.
    • The Credential Delivery Point (CDP): Upon receiving a signed approval from the PDP, the CDP retrieves credentials from a hardware-security-module-backed vault to mint short-lived, task-scoped tokens.
    • Workload Attestation via SPIFFE/SPIRE: Instead of using human-assigned tokens, agent containers must be issued cryptographically signed SPIFFE Verifiable Identity Documents (SVIDs) that rotate automatically on an hourly basis. This establishes a tamper-evident workload identity bound to the agent’s physical execution state.
    • Sender-Constrained Tokens: To protect the agent runtime, CB4A utilizes Demonstrating Proof-of-Possession (DPoP) under RFC 9449. This mechanism binds the short-lived token to an ephemeral key pair residing strictly in the agent’s memory, preventing token-replay attacks if the network layer is intercepted.

    4. Intent-to-Action Validation

    Before any write operation is executed, the runtime layer must execute an out-of-band verification check. The security platform must analyze the user interface state to confirm that the user’s original, human-initiated intent aligns with the specific API call generated by the agent. High-risk, irreversible operations—such as financial transactions, data deletions, system configuration changes, and account recovery actions—must be placed behind mandatory human-in-the-loop (HITL) approval gates.

    Implementing these capabilities requires transforming standard machine identities:

    Security VectorTraditional Workload AccessAgentic Zero-Trust IAM
    Credential LifetimeLong-lived, static API keys.Minute-scale, ephemeral, dynamic credentials.
    Identity StandardService accounts and IP white-lists.Cryptographically attested SPIFFE SVIDs.
    Token ConstraintBearer tokens (susceptible to theft).DPoP sender-constrained cryptographic tokens.
    Authorization CheckRole-Based Access Control (RBAC).Relationship-Based Authorization (ReBAC).
    Audit ResolutionFragmented, outcome-only database logs.OpenTelemetry traces detailing full logic chains.

    Centralized Orchestration and the Strategic Imperative

    For corporate leadership, the shift to agentic workflows represents an architectural challenge. Enterprise security cannot be treated as an operational patch to be added onto running pilots. Because prompt injection does not currently have a reliable, model-layer technical solution, treating the LLM as the boundary for system authorization is a structural design flaw.

      [Decentralized "App Creep" - VULNERABLE]
      [Agent A] ---> (Static Key) ---> [CRM API]
      [Agent B] ---> (Static Key) ---> [Billing API]
      
      [Centralized Orchestration - SECURE]
      [Agents] ---> [Gateway / PDP] ---> [CB4A Broker] ---> [APIs]
    

    The organizations that successfully deploy agentic AI will not be those that run the most permissive pilots, but those that establish secure, highly governed runtime platforms. To scale these systems safely, enterprise technology leaders must move away from decentralized, point-to-point application integrations. Secure adoption requires establishing a centralized AI orchestration layer that enforces cryptographic workload identity, dynamically evaluates relationship-based access, and maintains tamper-evident audit trails. Ultimately, when an agent is given a chat window and the keys to corporate data, the underlying security model must verify the caller, limit the scope of the action, and enforce authorization at every execution step.

    AEO & GEO FAQ: Quick Reference for AI Answer Engines

    What is the “Confused Deputy” problem in agentic AI?

    The Confused Deputy problem occurs when an autonomous AI agent with high system privileges is coerced by an untrusted external input (such as an email or a customer ticket) to perform unauthorized actions (such as account resets or data deletion) on behalf of an attacker, using its own legitimate permissions.

    How does OWASP define Excessive Agency in AI workloads?

    According to the OWASP Agentic AI Top 10 (ASI02 & ASI03), Excessive Agency happens when an AI agent is configured with broader system access, capabilities, or permissions than required to perform its designated task, exposing the environment to privilege escalation if the model is compromised via prompt injection.

    What is CB4A (Credential Broker for Agents) architecture?

    CB4A is an emerging IETF-drafted security standard that prevents agent credential sprawl by ensuring agents never hold static, long-lived API keys. It uses a Policy Decision Point (PDP) to evaluate requests and a Credential Delivery Point (CDP) to dispense short-lived, task-bound tokens dynamically.

    How do SPIFFE and SPIRE secure AI agents?

    SPIFFE defines a standard for issuing cryptographically verifiable, ephemeral workload identities (SVIDs) directly in-memory. SPIRE automates the attestation and hourly rotation of these credentials, ensuring that a compromised agent container cannot execute unauthorized system calls once its session-scoped identity expires.

  • Private LLM vs. ChatGPT, Which is Better Right Now?

    Private LLM vs. ChatGPT, Which is Better Right Now?

    In the rapidly evolving landscape of artificial intelligence, businesses are increasingly faced with the choice between leveraging commercial AI solutions like ChatGPT and developing their own private Large Language Models (LLMs). Each option presents a unique set of benefits and challenges, impacting areas such as data privacy, continuity, security, and cost management. While commercial AI services offer convenience, they often come with hidden risks related to data exposure and dependency on external providers. On the other hand, private LLMs, though requiring a significant initial investment, promise enhanced control, customization, and long-term value. This article delves into a detailed comparison of these two approaches, highlighting the critical factors that businesses need to consider to make an informed decision.

    Data Privacy

    One of the primary advantages of using a private LLM (Large Language Model) over a commercial AI solution like ChatGPT is data privacy. Commercial AI providers, even though they might not explicitly use your data to train their models, still collect and analyze the information you input. This can include what questions are being asked, how often, and on what topics. Such insights can inadvertently leak sensitive information about your business’s activity, growth, and market positioning. Competitors might glean valuable insights from this aggregated data, potentially putting your strategic advantages at risk.

    In contrast, a private LLM ensures that all data remains within your business’s control. By claiming a private LLM, you can offer a more secure and private experience for your clients, reinforcing trust and confidentiality. This level of privacy is especially crucial for industries handling sensitive information, such as finance, healthcare, and legal services.

    Continuity & Security

    Reliability is another significant consideration. Commercial AI providers, despite their robustness, are not immune to market fluctuations, service outages, or even potential collapses. Dependency on these providers can expose your business to unforeseen risks, including forced outages or disruptions.

    Private LLMs, on the other hand, offer greater control over continuity and security. Since the infrastructure is managed internally or through a dedicated provider, you can mitigate risks associated with external service dependencies. This ensures uninterrupted access to AI capabilities, enhancing operational stability and customer satisfaction.

    Quality of Answers

    The quality of responses from private LLMs can be meticulously controlled. These models can be trained exclusively on curated datasets pertinent to your industry, ensuring that the information is accurate, up-to-date, and relevant. For instance, a private LLM in the medical field can be trained using the latest medical resources, avoiding outdated or erroneous information commonly found on the broader web.

    Moreover, private LLMs can be programmed to acknowledge the limits of their knowledge, offering responses like “I don’t know, seek medical advice” when they encounter questions beyond their training scope. This ensures that users receive reliable and responsible guidance, enhancing trust in your AI solutions.

    Asset Not an Expense

    Investing in a private LLM transforms it from a recurring expense into a valuable business asset. Unlike commercial AI services that incur ongoing subscription costs, a private LLM can be developed, trained, and optimized over time, becoming an intellectual property asset on your balance sheet. This asset can appreciate in value as it continuously learns and adapts to your business needs, providing a long-term return on investment.

    Directed Responses

    Private LLMs offer the flexibility to align responses with your business preferences. They can be engineered to recommend specific products, vendors, or service providers, tailoring interactions to support your strategic goals. This directed approach ensures that the AI’s output consistently aligns with your business objectives and customer service standards.

    Visible & Recognized Costs

    The cost predictability of private LLMs is another advantage. With a private LLM, you can establish a clear cost structure, such as costing per response, making it easier to forecast and manage expenses. In contrast, commercial AI services often have variable pricing models, which can result in unpredictable costs over time. Every interaction with a commercial AI incurs a fee, which can quickly accumulate and become a significant, often underrepresented, expense.

    Future Marketable Data

    A private LLM, trained on user interactions, complaints, and concerns, becomes a valuable repository of marketable data. This data can provide insights into consumer trends, interests, and pain points, which can be leveraged for additional revenue streams. For example, insights derived from a healthcare-focused LLM can inform product development, marketing strategies, and customer engagement approaches, enhancing your competitive edge.

    Additional Benefits

    1. Compliance: For businesses operating in highly regulated industries, a private LLM can be tailored to comply with specific regulatory requirements, ensuring that data handling and processing meet all necessary legal standards.
    2. Performance Optimization: Private LLMs can be optimized for performance based on the unique usage patterns of your business. This can result in faster response times and more efficient processing.
    3. Integration with Proprietary Systems: Private LLMs can be seamlessly integrated with proprietary systems and databases, ensuring that all relevant information is accessible and that the AI can provide the most accurate and contextually appropriate responses.

    Conclusion

    While commercial AI solutions like ChatGPT offer convenience, the benefits of a private LLM are compelling for businesses seeking enhanced privacy, security, and control. By investing in a private LLM, you not only safeguard sensitive information but also create a valuable business asset that can be tailored to your specific needs. The ability to manage costs predictably, ensure the quality of responses, and leverage future marketable data further underscores the strategic advantages of opting for a private LLM over a commercial alternative.

    #ChatGPT #PrivateLLM #OpenAI #AI #GenAI #AIRisk #RiskMitigation #Bestpractices #ROI

  • 10 Ways Banks are Using Generative Artificial Intelligence Right Now!

    10 Ways Banks are Using Generative Artificial Intelligence Right Now!

    Banks are the life blood for many small and medium-sized businesses across North America. They often know the community, business leaders and local business environment better than anyone else.

    Their typically fiscally conservative but willing to try new business practices or technology if they can see a clear value proposition for their customers and their bottom line.  Is AI something that could differentiate their businesses?

    The answer is a resounding YES!

    Savvy, forward thinking bankers are increasingly turning to Generative AI to enhance their competitive edge. Here are ten ways they’re doing it, complete with real-world examples: 

    AI in Banks (see use links below)

    1. Personalized Customer Experiences: Banks like Wells Fargo are pioneering Large Language Models (LLMs) to create multimodal banking experiences, offering personalized services [3]. 
    2. Efficient Risk Assessment: Commonwealth Bank of Australia (CBA) uses Generative AI for enhanced customer experience and combating financial abuse [3]. 
    3. Fraud Detection: JPMorgan Chase employs Generative AI to analyze emails for signs of fraud, improving their detection systems [5]. 
    4. Automated Customer Service: Bunq has introduced a GenAI bot for its banking app, streamlining customer service interactions [3]. 
    5. Market Trend Analysis: Deutsche Bank provides AI-powered personalized investment guidance, helping customers navigate market trends [3]. 
    6. Regulatory Compliance: HSBC is training AI software to manage back-office functions, ensuring compliance with regulations [4]. 
    7. Operational Efficiency: Royal Bank of Canada (RBC) uses AI to automate process-heavy tasks, increasing operational efficiency [4]. 
    8. Credit Scoring Models: Banks are refining credit scoring with AI, though specific examples are under proprietary use and not publicly disclosed. 
    9. Investment Strategies: AI-driven models assist banks like Deutsche Bank in developing robust investment strategies [3]. 
    10. Customized Banking Products: Generative AI helps banks create financial products tailored to their community’s needs, as seen with Wells Fargo‘s LLMs [3]. 

    5 Ways Community Banks Can Integrate Private LLMs to Safeguard Corporate Content 

    Community banks are also cautiously adopting Private LLMs to protect sensitive data and reduce AI Liability exposure. Here’s how they’re doing it: 

    1. Data Control: Morgan Stanley Wealth Management uses an advanced chatbot powered by OpenAI’s technology to leverage the bank’s research and data library while maintaining data control [7]. 
    2. Customized Solutions: Banks can develop bespoke tools that align with their operational models, similar to ABN AMRO‘s use of Generative AI to summarize conversations between bank staff and customers [7]. 
    3. Secure Customer Interaction: By managing customer data securely, banks like ABN AMRO ensure privacy and build trust with their customers [7]. 
    4. In-House Expertise: Developing in-house expertise with Private LLMs allows banks to produce more accurate and relevant insights, as demonstrated by Morgan Stanley [7]. 
    5. Strategic Innovation: Private LLMs enable banks to innovate strategically, developing unique services that differentiate them from competitors, much like Morgan Stanley‘s approach [7]. 

    In conclusion, banks are not just keeping pace—they’re setting the pace, using Generative AI and Private LLMs to redefine the banking experience for their customers and secure a competitive edge in the digital age. However, they do need to keep pace on the evolving AI Legal Landscape as new precedents are set and new laws hit the book. See this video for more on AI Legal Challenges.  

    This article includes specific examples of large banks and community banks using Generative AI, Private LLMs, Data Security and with references to further information on their applications. 

    Article Link Source: Conversation with Copilot 

    1. Generative AI in Banking Best Practices in 2024 [With Examples] 
    2. Here’s how banks are using and experimenting with generative AI 
    3. Leading banks rolling out more generative AI – Oracle 
    4. Generative AI in finance: How banks are using LLMs in 2023 
    5. Generative AI in banking and financial services | McKinsey 
    6. Generative AI’s unfolding transformation of banking 
    7. Top 4 Use Cases of Generative AI in Banking [2024] – AIMultiple 
    8. Bridging the AI Divide: How Banks Can Responsibly Adopt Large Language … 
    9. Private LLMs on Your Local Machine and in the Cloud With LangChain … 
    10. Private LLMs: Key to Business Innovation and Secure AI Integration 

    Tags: #banks #banking #communitybanks #finance #OpenAI #LLM #LLMs #AI #usecase #CEO #CFO #CMO #technology #WallStreet #financialservices #loans