Introduction
This post is focused on LLM/Agentic features and ideas we would like to implement in eramba. We have two approaches to this:
- Prompt based LLM
- Agentic Employees
Prompt based LLM
This is an LLM Integration that allows the GRC team (and others in the company) to ask questions such as: What Risks overlap? What changed in between policies? How policy XCY helps to us to be compliant? Is that Policy implemented and tested? Create a chart with Risks per Business Units.
Where the user asks questions:
- Inside eramba we include options (buttons) such as “Review this Policy for Compliance”
- Inside eramba we include a chat, where you click the item (that gives you context) and then you chat with a prompt a set of pre-defined options for that item type (policy, control, etc)
- Over an LLM client, the benefit is you can ask questions that go across a specific item or module. The problem is that the output is non-structured most of the times.
Scope and Security
When questions are done inside eramba, the scope of authorization and visualisation is already controlled by eramba RBAC.
When questions are done from an LLM client, we need to transport that identity to eramba before giving out answers, the alternative is to give this type of access uniquely to administrators. Write operations are tricky here, there is no frontend validation and this is as we know tricky.
Agentic Employees
This is an Agentic autonomous tasks where an agent takes care of tasks continuously, for example handle the review of a policy and the interactions required to do that with people in other departments. Agents will be implemented in phases:
- Passive agents that observe and report
- Semi-autonomous agents that recommend actions
- Only then: limited autonomous remediation.
We define tasks somewhere, we run a test to see what the LLM does, we let the LLM do the job on their own and report when it faces a challenge. There is a ton of authorization and visualization work to be done here to make sure this agent does not do crazy things.