Performance evaluation of plagiarism detection method based on the intermediate language Vedran Juričić Tereza Jurić Marija Tkalec

Plagiarism detection methodPlagiarism detection method Method for detecting plagiarism in source code for.Net languages C# Visual Basic.Net C++ … Identify similar code fragments Determine similarity between source files Based on intermediate language 2

Plagiarism detectionPlagiarism detection 3 1. using System.Text; 2. namespace Test { 3. class Math { 4. public double GetMaximum(double[] Input) { 5. double result = Input[0]; 6. foreach (double temp in Input) { 7. if (temp>result) 8. result = temp; } 9. return result; } } } 1. using System.Text; 2. namespace Test { 3. class Math { 4. public double GetMaximum(double[] Input) { 5. double result = Input[0]; 6. for (int i=0;i<Input.Length;i++) { 7. if (Input[i]>result) 8. result = Input[i]; } 9. return result; } } } Similarity = Number of overlapping lines / Total number of lines = 6 / 9 = 66,66% FirstSecond

But… 4 1. using System.Text; 2. namespace Test { 3. class Math { 4. public double GetMaximum(double[] Input) { 5. double result = Input[0]; 6. foreach (double temp in Input) { 7. if (temp>result) 8. result = temp; } 9. return result; } 1. using System; 2. namespace OtherTest { 3. class MyClass { 4. public double ReturnMaximum(double[] Array) { 5. double current = Input[0]; 6. for (int j=0;j<Input.Length;j++) { 7. if (Input[j]>current) 8. current = Input[j]; } 9. return result; } Similarity = Number of overlapping lines / Total number of lines = 0 / 9 = 0,00% FirstSecond

Problems Modification of variable names, types, constants Modification of class member definitions Line and command reordering … Solution Detail analysis Complex preprocessing For each supported language 5

Our solutionOur solution Convert from source language to low-level language (Common Intermediate Language) By using existing tools Compiler Disassemler Tools exist for all.Net languages 6

Our solutionOur solution 7 using System.Text; namespace Test { class Math { public double GetMaximum(double[] Input) { double result = Input[0]; foreach (double temp in Input) { if (temp>result) result = temp; } return result; }.method public hidebysig instance float64 GetMaximum(float64[] Input) cil managed { // Code size 61 (0x3d).maxstack 2.locals init (float64 V_0, float64 V_1, float64 V_2, float64[] V_3, int32 V_4, bool V_5) IL_0000: nop IL_0001: ldarg.1 IL_0002: ldc.i4.0 IL_0003: ldelem.r8 IL_0004: stloc.0 IL_0005: nop IL_0006: ldarg.1 IL_0007: stloc.3 ….. IL_0037: ldloc.0 IL_0038: stloc.2 IL_0039: br.s IL_003b IL_003b: ldloc.2 IL_003c: ret } // end of method C::GetMaximum C# language Common Intermediate Language C# compiler nop ldarg.1 ldc.i4.0 ldelem.r8 stloc.0 nop ldarg.1 stloc.3 … ldloc.0 stloc.2 br.s ldloc.2 ret

Plagiarism detection systemPlagiarism detection system Evaluate the performance Analyze and compare behavior to most commonly used plagiarism detection systems: MOSS JPlag CodeMatch 8

Tested systemsTested systems MOSS Developed in Commonly used in computer science faculties Supports 26 programming languages JPlag Developed in Commonly used in education Supports C, C++, C# and Java 9

Tested SystemsTested Systems CodeMatch Developed in Commercial software Supports 26 languages ILMatch (our system) Developed in Supports all.Net languages (currently 59 languages) 10

Testing 6 test categories 50 test cases covering common code modification techniques Evaluation methods Precision, recall F-measure 11

Results 12 MOSSJPlag CodeMatchILMatch Highest F-measures

Positive No impact User comments Code formatting Modification of variable and class names Modification of class members Changing data types Some impact Replacing expressions and loops Rewritting code in different language 13

Further workFurther work Significant impact Reordering operands Reordering class members Adding redundant statements and variables Improvements in comparison algorithm 14