The task of AI agents is evolving. They are no longer used for conversations and recommendations but take on real-world business tasks such as document processing, updating customer records, performing operations, monitoring, and business process management. However, the ability of AI agents to take real-world actions introduces potential security, accuracy, compliance, and human control risks.
Businesses can use ML as a service to build intelligent agents while retaining the ability to control the tasks they perform. The combination of AI and necessary restrictions can be achieved by defining agent responsibilities, implementing permission-based access, and introducing human oversight.
An AI agent should have a set of permitted operations it can perform and denied operations it cannot perform. For instance, AI agents can be allowed to sort customer requests, schedule meetings, and automatically generate documents. However, monetary transactions, record deletion, and contract approval should require human supervision. It is the responsibility of the organization to hire AI agents to establish the former and the latter lists.
The list of permitted operations should minimize the chances of agent error and unauthorized AI agent actions. Businesses should also define expectations and instructions so that the agent knows what it should do and when to ask for help.
An AI agent should not have free and unfettered access to business applications. Through role-based permissions, authentication, and API restrictions, organizations can provide agents with only the data and tools they need to perform their tasks. Introducing the authorization step for each operation an AI agent wants to perform creates an additional layer of security validation. For instance, for sensitive operations, an employee can review and approve an action before it is completed.
Organizations should also update the set of permitted operations for an AI agent based on the evolving needs of the business. Overall, restricted access helps reduce the likelihood of an agent performing an incorrect operation or a malicious third party influencing the agent to disclose sensitive information.
Like any information system, AI agents can produce erroneous results if the information they use is faulty. By connecting AI agents to databases, knowledge bases, APIs, and business applications, organizations can ensure that the information used by agents is accurate and relevant. However, it is also critical to ensure that an AI agent cannot use information not intended for use. For instance, businesses should provide agents with access to only those databases and applications that have been approved.
Custom chatbot services can provide similar restrictions for conversations. In this way, an organization can ensure that information received from users is relevant and that the answers provided by the chatbot are based on trusted sources.
Human error is unavoidable, and therefore businesses should ensure that decisions with significant financial, legal, customer, or compliance implications have the approval of a human. A human-in-the-loop design involves an employee in making decisions about when an AI agent’s judgment is sufficient to proceed or if the situation requires a person to make decisions.
Such a control mechanism helps ensure that businesses retain responsibility for decisions made by AI agents. As organizations begin to use AI for day-to-day operations, the human-in-the-loop model provides an additional layer of security. Additionally, businesses can define confidence thresholds below which an AI agent must ask for help from a person. For instance, an AI agent can handle a large volume of repetitive requests automatically, and only those requests that do not meet specific confidence thresholds should require human review.
Businesses should keep logs of all actions, including the information used and decisions made by the AI agent, tools involved, and specific actions taken. Through such logging, organizations can monitor the actions of the AI agent and detect any questionable patterns, errors, or attempts by third parties to compromise the system. Security measures should also protect against such attacks as prompting, manipulation, and unauthorized use of authentication credentials. Additionally, an AI agent should have a set of actions that it must perform if it cannot complete a task or reaches an unexpected result.
Moreover, organizations should test and continuously improve the system by analyzing how often an agent takes certain actions. For instance, before using an AI agent in production, an organization can test it with unexpected or unknown data to see how the model will respond and what decisions it will make.
An AI agent should not perform an action just because it has been asked to. Instead, organizations should add validation steps to determine whether an action is appropriate before the application or system actually performs it. For instance, an AI agent that automatically processes purchase orders can first check the quantity and price to ensure that they are appropriate and that the purchase does not exceed the established budget.
If the purchase order does not meet the established criteria, an employee should review the request. This validation step is particularly important for the accuracy and security of applications that involve financial, customer, or operational data.
By gradually increasing the scope of work performed by an AI agent, organizations can ensure that agents operate within acceptable limits without introducing unacceptable risks. Initially, an AI agent should focus on processes with low error and impact risk.
Using ML as a service, organizations can launch AI agents, monitor, and update their performance without having to manage every detail of the model. Additionally, before launching AI agents across departments, an organization can first use them for narrowly defined tasks. Moreover, based on the results obtained, the organization can determine in which other areas of the business the use of AI agents makes sense.
AI agents are powerful tools that can help businesses operate more efficiently by taking on some tasks typically performed by people. However, realizing the benefits of using AI agents requires a coordinated effort to address issues of free will, accuracy, and security. By carefully defining the responsibilities of AI agents, introducing permission-based access, keeping humans in the loop, validating results, and monitoring and measuring performance, businesses can scale the use of AI agents safely and securely.
Essentially, businesses should not launch AI agents that operate independently but create systems in which AI agents understand their limitations, follow rules, seek help when necessary, and perform only those actions that have been approved.