Combinatorial Testing on ID3v2 Tags of MP3 Files Zhiqiang Zhang1, Xiaojian Liu2, Jian Zhang1 1 Institute of Software, Chinese Academy of Sciences 2 Institute of Automation, Shandong Academy of Sciences

Introduction MP3 (MPEG-1 or MPEG-2 audio layer III) is an audio file format developed by the Moving Pictures Experts Group (MPEG) One of the most popular audio formats Supported by almost all audio players Usually, an MP3 file comes with an ID3 tag, which stores audio information such as: Title, artist, album, … ID3 has two unrelated versions: ID3v1 & ID3v2

Overview ID3v2 tag format Test goals Modeling Experiments & Results Conclusion

Combinatorial Testing Considerably high fault coverage Model the system under test (SUT) as a parameterized black box Usually use a covering array as the test set Small # of test cases

MP3 frames & ID3v2 tags An MP3 file is built up from a sequence of MP3 frames (MPEG frames) A short segment of audio data The first 11 or 12 bits are always set, can be used for synchronization Sometimes MP3 frames depend on each other, and cannot be freely cut ID3v2 tag: A metadata container for audio file information Located at the beginning of an MP3 file ID3v2 tag MP3 frames

ID3v2 tag format In our work, we consider a subset of the ID3v2 tag format An ID3v2 header Several ID3v2 frames ID3v2 header format: ID3v2/file identifier “ID3” ID3v2 version $03 00 ID3v2 flags %abc00000 ID3v2 size 4 * %0xxxxxxx ID3v2 header ID3v2 frame 1 ID3v2 frame 2 ID3v2 frame 3 … Tag size

ID3v2 Frames An ID3v2 frame consists of a frame header and the frame content Frame header Frame ID $xx xx xx xx (four characters) Size $xx xx xx xx Flags $xx xx Text information frames Text encoding $xx Information <text string> Frame header Frame content Frame size

Text Encodings Encodings commonly used in China: $00: ASCII or GBK $01: Unicode (UTF-16) Little-endian (LE) Big-endian (BE) $02: Unicode BE $03: UTF-8

Test Goals Usually, tag processing and audio playing are performed by separate modules of MP3 players Audio playing is often done by a decoder (out of the scope of our work) ID3v2 tag processing Goal I: text information recognition and display Text encodings of some frames may influence the processing of other frames Goal II: robustness against bad header & frame sizes Offset related May cause buffer overflow or vulnerable read operations

Modeling (Test Goal I) Six types of text information frames The model: TIT2 (title), TPE1 (artist), TALB(album), TRCK (track number), TCON (genre) and TYER(year) The model: TIT2 TPE1 TALB TRCK TCON TYER NONE ASCII GBK ULE UBE UBE2 UTF-8

Modeling (Test Goal II) The sizes indicated by the ID3v2 header & frame size bytes might be faulty May cause critical failures For building each test case Set the header and frame size bytes Set the actual header and frame sizes Build an ID3v2 frame Build an ID3v2 tag by filling the content with frames Attach MP3 frames

Modeling (Test Goal II) Building an ID3v2 frame Building an ID3v2 tag Frame size bytes Frame header $00 AAAAAA… Tag size bytes ID3v2 header ID3v2 frame … Actual frame size Actual tag size

Modeling (Test Goal II) Building a complete MP3 file Attach a sequence of MP3 frames after the ID3v2 tag In some cases, the actual tag size is not an integral multiple of the total frame size To fill the remaining space: 4 filling modes

Filling Modes None Safe Cut Overflow Tag size ID3v2 frame MP3 frames $00 00 00… Tag size ID3v2 frame MP3 frames ID3v2 frame MP3 frames Tag size None Safe Cut Overflow

Modeling (Test Goal II) TS TSD FS FSD FM ATCHMP3 0KB 0B NONE YES 4KB 1KB 256B 64B SAFE NO 64KB -1KB -64B CUT OVFL TS: tag size indicated by the ID3v2 header size bytes TSD: difference of the actual tag size compared with TS FS: frame size indicated by the frame header size bytes FSD: difference of the actual frame size compared with FS FM: filling mode ATCHMP3: whether MP3 frames are attached or not

Modeling (Test Goal II) Normal constraints IF TS=0KB THEN TSD<>-1KB IF FS=0KB THEN FSD<>-64B When TS=0KB and TSD=0KB, the actual tag size is 0KB. So no frames will be filled. Thus FS, FSD and FM are invalid parameters Introduce a special parameter value ‘#’ for invalid parameters IF TS=0KB AND TSD=0KB THEN FS=# AND FSD=# AND FM=# IF TS<>0KB OR TSD<>0KB THEN FS<># AND FSD<># AND FM<>#

Experiments Test subjects: Test generation: I: an on-vehicle leisure and entertainment system II: a portable MP3 player Test generation: Use Microsoft’s PICT Test Goal I: 57 test cases (strength=2) Test Goal II: 59 test cases (strength=3)

Experimental Results (Test Goal I, Subject I) Condition Results ALL ASCII text is correctly displayed. UBE2 text is incorrectly displayed.* TIT2=NONE Title is displayed as file name. TPE1=NONE Artist is displayed as folder name. TPE1{ASCII,UBE2} AND TIT2{GBK,ULE,UBE,UTF-8} Title is not displayed.* TPE1{ASCII,UBE2} AND TIT2{GBK,ULE,UBE,UTF-8} Title is displayed as file name.* Artist is displayed as folder name.* 2 passed, 55 failed Actually, only ASCII text can be displayed

Experimental Results (Test Goal I, Subject II) Condition Results TIT2{UBE2,UTF-8} File name is displayed as the title.* TIT2<>NONE AND TPE1{UBE2,UTF-8} OTHERWISE All information is correctly displayed. 33 passed, 24 failed Problem when processing two encodings Failures caused by 1-2 parameters

Experimental Results (Test Goal II, Subject I) Condition Results TS=0KB AND TSD=1KB AND FS<>0KB AND ATCHMP3=NO The subject encounters a failure, and all audio files copied into the subject cannot be played.* ATCHMP3=YES The file can be successfully played. The file is skipped. 52 passed, 7 failed Failures caused by 4 parameters

Experimental Results (Test Goal II, Subject II) Condition Results ATCHMP3=YES The file can be successfully played. ATCHMP3=NO The player shows that the file is damaged and proceeds to the next file. All test cases passed We occasionally found a bug When ATCHMP3=NO, the subject Displays that the file is damaged for a few seconds Proceeds to the next file. Just between this two moments, it is not possible to play back to the previous file Not Discovered by CT!!

Conclusion Benefit from CT: Small test suite Test Goal I: 57 out of 7323=2744 exhaustive test cases Test Goal II: 59 out of 3442=648 exhaustive test cases (including test cases not satisfying the constraints) Interaction faults discovered Failures are caused by 1-4 parameters

Additional Results Experiments are conducted on two PC audio players PC audio player I perfectly passes all the tests PC audio player II: Test Goal I: similar failures as subject I Test Goal II: When all test cases are added to the player playlist, the player’s UI begin to respond slowly The player got stuck when playing 4 test cases

Thank you！