Feature - history feature / activity log

kisero · August 2, 2023, 1:25pm

We will be re-working the history functionality:

We tracked down multiple requests for this feature:

Also some Github stuff:

General

We will need to build a new history functionality. We want to call this Activity Log because we want to log all activities on ant item:

changes on the fields of the item
changes on the stats of the item
notifications triggered by the item

I’m thinking this feature will be called “Activity Log” and will become a new sub-tab on all modules OR one more “Common Feature”, most likely from a UI perspective, i think is better if we think of this as a common feature (right next to trash).

We will keep the item “History” button, but that will just redirect you to the activity log with a filter that shows log entries for that item.

Special Attention

changes on custom fields must be logged
changes on risk score must be logged
changes on risk classifications must be logged
changes on “Content” policy must be logged and easily displayed

Leverage Existing Features

When you load this new common feature we need the standard features for any module, in particular:

Filters: search anything across changes on any item
Reports: build reports about changes done in the module
Dynamic Status: label changes you want to know about
Notifications: send emails when certain conditions happen

Current History Button

The “History” item functionality we have today will be a modal (or just load the index) as today that will load the tab Activity Log module data (filter) with a parameter to this single item.

Default Columns (Index)

The filter will show the following columns (I MIGHT BE MISSING THINGS!)

General: Who (is a users or groups)
General: When (is a date picker with (-10, etc) too)
General: Activity Type (drop down with the following options Add, Edit, Delete, Restore, Status Change, Comment, Attachment, Notification)
Item ID: we need to know the id we are tracking, we do not have always a name for the item so we need the id here
Parent ID: we are sometimes tracking reviews or child items, then we also need this. Because when you load the reviews changes you will need to know what changes happened on the parent id.
Old Field Value:
New Field Value: Name (this will list all fields and you can search changes for a specific field)
Dynamic Status: Name (you can search against a dynamic status)
Notification: Type (you can select by notification type)
Notification: Recipient (search by who sent this)

ref: https://docs.google.com/spreadsheets/d/1eJGyTzPHdCJ5K7KycptWnjDDdOUo1nbkvGX4uw2udDE/edit#gid=0

Retention

We need to keep an eye on the number of records this creates, for that we need a new system setting where for every module we define:

Retention (100 rows) or time (last 50 days, no retention, Etc)

by default, i would disable logging in multiple modules.

Parent/Child Relationship

A policy has one or more reviews, both type of items need an activity log. The review needs to be able to tell what changes its parent went through at any time. This is a key new functionality.

david.schroth · August 15, 2023, 11:21am

Throwing in my two cents here on retention (well, 1.83 Euro cents based on today’s exchange rate) -

One of my clients has some regulatory requirements that requires keeping a much longer history - to the point that they’d likely prefer to maintain all the logs and not purge after some number of days or rows.

Separately, when doing contract reviews and questionnaires for my clients that are SaaS platforms, this issue comes up a couple of ways - 1. Asking about retention time (never about number of records) with the expectation that it is in years (often 3 years) and 2. Asking if they are able to get a feed of the logs for their SIEM (this is more often banking/finance/healthcare industry).

Given the above, I would suggest architecting the change to be able to handle logging all the things and retaining forever (especially for on prem), then from a SaaS perspective, perhaps you’ll have some limit for log storage (e.g. 1 or 3 years) that’s included with the service.

HamzaG · August 16, 2023, 7:44am

Hi,
i would like to add a something: for the moment, the history field doesnt take in count the custom fields and the values encoded in it.

kisero · August 16, 2023, 7:46am

Yes is true, is also part of the feature

kisero · October 5, 2023, 11:20am

The feature will be rolled out on the next release, we have been testing it and it seems ok.

Both History and Audit Log features will coecist for some time (a year most likely):

You can access Activity Log from the “Item” or the “Menu”

When you access you see all possible changes: fields, dynamic status, notifications triggered, Etc. You can also use common features (Filters, Reports, Notifications, Status).

You can for example:

search all changes done yesterday and email them to you
search all changes where field “group” is “admin” and track down creation of users under “Admin”

You can track down changes on “parent” and “child” items. This is very important for the comming “Review” upgrade. You will be able to track down, on every review , what changes happened on the parent item.

Retention is limited to 60 days for now, we will extend this to a year or more based on how the performance of the feature turns out:

kisero · February 8, 2024, 12:44pm

We need to expand retention options on settings to include:

180 Days
365 Days
500 Days

By default, we need to force all systems to have 500 days selected by default. Testing of the feature for the last month shows around 15 Mb of logs per month, so it should be ok to rotate logs every 12-13 months.

The rotation of logs needs to work in a way that logs older than 500 days needs to be removed daily.