Measuring Privacy Compliance with Process ... - MetriSec 2011

Measuring Privacy Compliance with Process Specifications

1/ 26

Measuring Privacy Compliance with Process Specifications Sebastian Banescu and Nicola Zannone Eindhoven University of Technology

September 21, 2011

Sebastian Banescu and Nicola Zannone



2/ 26

Outline Motivations Conformance Metrics State of the Art Comparative Study Privacy Factors Measuring Infringements Tool-Supported Compliance Verification Conclusions & Future Work




3/ 26

Motivations

Motivations Privacy and Data Protection

Unpredictable Domain

Stringent requirements on collection, processing and disclosure of personal data

Impossible to know in advance when and where exceptions occur

I EU Directive 95/46/EC, HIPAA

I Healthcare domain I In emergency situations there may

occur deviations from specifications

I I

Organizations must adopt flexible security measures Existing solutions are too rigid!! I I I

Prevent bad things happen Not possible to define all possible exceptions in advance In emergency situation security mechanisms are bypassed (break-the-glass) Sebastian Banescu and Nicola Zannone



4/ 26

Conformance Metrics

Measuring Privacy Compliance Goal: Automate auditing process to assess privacy infringements I I I

Hold users accountable for their behaviour Not every deviation from specifications has the some severity Compliance metrics for measuring “privacy” divergency from specifications




5/ 26

Conformance Metrics Example

Example: Process Specification & Audit Trail #

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9

Alice Bob Bob Charlie Bob Bob Bob Alice Alice

R D D L D D D R R

R01 D01 D02 L01 D03 D04 D05 R03 R04

{ID, MedRec/ Demographic}

I

I I

Tasks are rounded rectangles Gates are diamonds with a sign Data items are rectangles with folded corner

An audit log capture user behavior I I I I


History} History/ TestResults} History} History/ Prescription} History} History/ Prescription} Health Insurance}

BPMN specification I

I

{MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/

user who performed the task role held by the user task that has been executed data that has been accessed



6/ 26


Example: Audit Trail 1


#

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9


R D D L D D D R R

R01 D01 D02 L01 D03 D04 D05 R03 R04

{ID, MedRec/ Demographic} {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/




6/ 26




#

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9


R D D L D D D R R

R01 D01 D02 L01 D03 D04 D05 R03 R04





6/ 26


Example: Audit Trail 1 #

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9


R D D L D D D R R

R01 D01 D02 L01 D03 D04 D05 R03 R04



Compliant




7/ 26


Example: Audit Trail 2 #

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9 10 11 12

Alice Bob Bob Charlie Bob Bob Bob Bob Bob Bob Alice Alice

R D D L D D D D D D R R

R01 D01 D02 L01 D03 D04 D02 D03 D04 D05 R03 R04


I


{MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/

History} History/ TestResults} History} History/ Prescription} History} History} History/ Prescription} History} History/ Prescription} Health Insurance}

User repeated the execution of some tasks



8/ 26




#

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9

Alice Bob Bob Charlie Bob Bob Alice Alice Alice

R D D L D D R R R

R01 D01 D02 L01 D03 D04 D05 R03 R04



I

User with a certain role executes the task designated for another role

I

Unauthorized access to information



9/ 26




#

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9


R D D L D D D R R

R01 D06 D02 L01 D03 D04 D05 R03 R04

{ID, MedRec/ Demographic} {MedRec/ Financial} {MedRec/ History} {MedRec/ History/ TestResults} {MedRec/ History} {MedRec/ History/ Prescription} {MedRec/ History} {MedRec/ History/ Prescription} {MedRec/ Health Insurance}

I

Untrustworthy user executes a task that was not in the specification

I

Unauthorized access to information



10/ 26

Conformance Metrics State of the Art

Conformance Metrics Notation I

Not every deviation from specifications has the same severity

I

Measuring severity of infringements Existing conformance metrics

I

I I I

I

Binary metrics Sequence distance metrics Process fitness metrics

Measures based on the performed activities


I P - set of business processes I Σ - set of observable tasks in P I A trace is a finite sequence over Σ I Σ∗ - set of traces over Σ I T (p) ⊆ Σ∗ - set of traces generated by a process p ∈ P I d : P × Σ∗ → R ∪ {∞} - process metrics I d : Σ∗ × Σ∗ → R ∪ {∞} - sequence metrics I · - empty trace I ασ - concatenation of traces α, σ ∈ Σ∗



11/ 26

Conformance Metrics Binary Metrics

Binary Metrics

I I

Simplest type of metric for conformance checking Return a positive answer only if the audit trail fully conforms to the specification I I

aσ - trace representing the expected behavior bσ 0 - audit trace representing actual behavior

 0    ∞ 0 dB (aσ, bσ ) = dB (σ, σ 0 )    ∞


if if if if

aσ = · and bσ 0 = · aσ = · xor bσ 0 = · a=b a 6= b



12/ 26

Conformance Metrics Sequence Distance Metrics

Sequence Distance Metrics

Take two sequences as input and count the number of changes needed to transform one sequence into the other I

Suppressing Distance

I

Replacing Distance

I

Levenshtein Distance




13/ 26


Suppressing and Replacing Distance Suppressing Distance I

Count the number of actions that must be removed from the audit trace to obtain the expected behavior

Notation I aσ - trace representing

 ∞    |bσ 0 | 0 dS (aσ, bσ ) = d (σ, σ 0 )    S 1 + dS (aσ, σ 0 )

if if if if

aσ 6= · and bσ 0 = · aσ = · a=b a 6= b

the expected behavior I bσ 0 - audit trace

representing actual behavior

Replacing Distance I

Count the number of replacements necessary to obtain one trace from the other  0    ∞ 0 dR (aσ, bσ ) = d (σ, σ 0 )    R 1 + dR (σ, σ 0 )

if if if if

aσ = · and bσ 0 = · aσ = · xor bσ 0 = · a=b a 6= b




14/ 26


Levenshtein Distance Notation Count the number insertions, suppressions and replacements needed to obtain one trace from the other

I aσ - trace representing

the expected behavior I bσ 0 - audit trace

 |bσ 0 |    |aσ| dL (aσ, bσ 0 ) = dL (σ, σ 0 )    1 + min(dL (σ, σ 0 ), dL (aσ, σ 0 ), dL (σ, bσ 0 ))


representing actual behavior

if if if if

aσ = · bσ 0 = · a=b a 6= b



15/ 26

Conformance Metrics Process Fitness Metrics

Process Fitness Metrics I I

Use process structure rather than comparing traces Simulate audit trace against process model I I

I I I

Simulation is accounted for using tokens A token is produced by a start event, and as a result of the execution of a task A token is consumed by the execution of a task or by an end event A token is remaining if not consumed A token is missing if artificially created to allow the execution of the process r 1 m + dF (P, l) = 2 c p I I I I

m is the number of missing tokens r is the number of remaining tokens c is the number of consumed tokens p is the number of produced tokens




16/ 26

Conformance Metrics Comparative Study

Comparing Conformance Metrics Apply the previously presented metrics to 4 audit trails shown earlier: I

Audit Trail 1: Correct execution Audit Trail 2: Doctor repeats some tasks

I

Audit Trail 3: Recepionist executes a task of a doctor

I

Audit Trail 4: Doctor executes a task not specified in process model

I

Audit Trail

T

dB TR

T

dS TR

T

dR TR

T

dL TR

dF T

1 2 3 4

0 ∞ 0 ∞

0 ∞ ∞ ∞

0 3 0 ∞

0 3 ∞ ∞

0 ∞ 0 1

0 ∞ 1 1

0 3 0 1

0 3 1 1

0 1/10 0 1/7

Table: Severity of infringement for the audit trails




17/ 26

Privacy Factors

Privacy Factors I

Additional information allows a better discrimitation of infringements

I

Idea: Use multiple factors for measuring privacy compliance

I

Information should be available somewhere!!

In the audit trail: I

personal data used in the process

I

users accessing and processing data

I

actions that users perform

#

User

Role

Task

Data Items Accessed

1 2 3 4 5 6 7 8 9


R D D L D D D R R

R01 D01 D02 L01 D03 D04 D05 R03 R04



{MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/ {MedRec/




18/ 26

Privacy Factors Data Factors

Data Factor I

Personal data items have different sensitivity levels

I

Sensitivity differs from an individual to another (subjective)

I

Assign privacy weights to data items

I

p ∈ R+ represents the privacy weight for disclosing personal data




19/ 26

Privacy Factors User Factors

User Factor: Reputation I

I

I I

Infringements caused by untrusted users present a high risk of privacy loss Reputation measures the trustworthiness of users based on their past behaviour Reputation used to amplify the severity of infringements r ∈ [0, 1] is the reputation of the user performing the task I

1 means very good and 0 very bad




20/ 26

Privacy Factors User Factors

User Factor: Role I I I I

Roles used to define function and responsibilities of users Access rights assigned to roles Risk level depends on the role held by the user sR ∈ [0, 1] is the semantic distance between the role held by the user and the role defined in the specification I I

0 means that the roles are semantically equivalent 1 means that the roles are completely incompatible




21/ 26

Privacy Factors Action Factors

Action Factor I

Task performed by a user during the execution of the process I I

I

I

Only factor used in existing conformance metrics Every deviation counts equally

A more accurate measurement of the severity of infringements requires taking into account which tasks have been executed sT ∈ [0, 1] is the semantic distance between the task in the specification and the task actually executed I I

0 means that the tasks are semantically equivalent 1 means that the tasks are completely incompatible




22/ 26

Measuring Infringements

Measuring Infringements Idea: replace the constant factor 1 in sequence distance with the extent of the deviation obtained using the privacy factors Φ(a, b) = (c1 − c2 r )[(1 + c3 sR )(1 + c4 sT )(1 + c5 p) − 1]

Notation I ci ∈ R+ , i ∈ {1, ..., 5} are constants I

privacy penalty due to unauthorized access to data X p= δi δi ∈B\(A∪Ku )

I

c1 > c2

I A: the set of data items that a user

is allowed to access I B: the set of data items accessed

during the actual task execution I Ku : the set of data items previously

accessed by user u




23/ 26


Revisiting Levenshtein Distance

Severity of an infringement can be assessed by combining the Levenshtein distance with metric Φ

 Φ(·, b) + dLΦ (aσ, σ 0 )    Φ(a, ·) + dLΦ (σ, bσ 0 ) dLΦ (aσ, bσ 0 ) =  dLΦ (σ, σ 0 )   Φ(a, b) + min(dLΦ (σ, σ 0 ), dLΦ (aσ, σ 0 ), dLΦ (σ, bσ 0 ))


if if if if

aσ = · bσ 0 = · a=b a 6= b



24/ 26


Example I

Audit Trail 1: Correct execution

I

Audit Trail 2: Doctor repeats some tasks

I

Audit Trail 3: Recepionist executes a task of a doctor

I

Audit Trail 4: Doctor executes a task not specified in process model

I Semantic distance (for both roles and tasks) calculated Audit Trail

T

dL TR

dL

1 2 3 4

0 3 0 1

0 3 1 1

0 0.34 11.28 18.8

using Latent Semantic Analysis

Φ

I

a semantic relatedness metric which uses a high-dimensional linear associative model to assess similarity of words

I

implementation by the Rensselaer MSR project I Privacy weights given by user I Reputation given by reputation system I c1 = 1.1, c2 = c3 = c4 = c5 = 1




25/ 26

Tool-Supported Compliance Verification

Tool

I

Inputs I

I

I I

I

Generates all possible valid traces of the process model I

I

Process specification generated using BPMN Modeler Eclipse plugin Audit log in eXtensible Event Steam (XES) Constants in metric Φ Domain knowledge

No loops

Evaluates the audit log against these traces using the selected metrics




26/ 26

Conclusions & Future Work

Conclusions & Future Work

I

In existing conformance metrics every deviation from specifications counts equally

I

Identified a number of factors to quantify severity of infringements

I

These factors can be accommodated into sequence distance metrics

I

The proposed privacy metric is extremely flexible allowing the specification of preferences and weights for each factor

I

Some remaining issues I I

Deal with loops Define constants in metric Φ

