Transcript Measuring virtual machine detection in malware using DSD tracer

Boris Lau, Vanja Svajcer
Sophoslabs, Journal in Computer Virology, 2008
報告者：張逸文
Outline
 Introduction
 Virtual machine detection methods
 Methodology of our study with DSD-Tracer
 Results
 Conclusion
2
Introduction（#1）
 Virtual machine technology is first implemented by IBM
 More attention from virus writers & computer security
researchers
 If in VM，malware will behave like a normal program
 If the proportion is > 0.1%，developing an environment
to successfully analyze VM-aware malware is important
3
Introduction（#2）
 The most common security use cases with VM
 Software vulnerability research
 Malware analysis
 Honeypots
4
Virtual machine detection methods（#1）
 If VM is detected, the malware will
 stop its execution or
 launch a specially crafted payload
 Zlob Trojans
 IRC bots
 Executable packers
5
Virtual machine detection methods（#2）
 Detection of running under MS virtual PC using VPC
communication channel
 Communication between guest OS & VMM
 Exceptions due to opcode：0x0f, 0x3f / 0x0f, 0xc7, 0xc8
 Call different VMM services： 0x07, 0x0B
6
Invalid instruction VPC communication channel detection
7
Virtual machine detection methods（#3）
 Detection of running under VMware using VMWare
control API
 VMWare backdoor communication
 guest ↔ host communication
 IN instruction
 port 0x5658
 eax：0x564D5868（VMXh）
 ebx ：function number
8
9
Anti-VMWare prevention virtual machine
initialization settings
10
Virtual machine detection methods（#4）
 Redpill（using SIDT, SGDT or SLDT）
 SxxT x86 instruction
 Return the contests of the sensitive register
 IDT in VMWare is 0xffXXXXXX
 IDT in Virtual PC is 0xe8XXXXXX
 Compare with 0xd0
 Invalid in multi processor system
11
Redpill
12
Virtual machine detection methods（#5）
 SMSW VMWare detection
 Store Machine Specific Word instruction
 Return 16-bit result
 32 bits register（16-bit undefined + 16-bit result）
 In VMWare, the top 16-bits doesn’t change
13
SMSW VMWare detection code
14
Methodology of our study with
DSD-Tracer（#1）
 DSD-Tracer
 identify obfuscation packers
 dynamic & static analysis
15
Methodology of our study with
DSD-Tracer（#2）
16
Methodology of our study with
DSD-Tracer（#3）
 Dynamic component
 Instructions decoded before its execution
 All CPU registers
 Reads / writes to virtual / physical memory
 Interrupts / exceptions generated
 Instrumented virtual machine
 Low-level information
17
Methodology of our study with
DSD-Tracer（#4）
 Static component
 C++ interface
 Python Script
 Match known techniques for detecting VM
 Automatic replication harness
 Web-based automatic replication harness
18
Methodology of our study with
DSD-Tracer（#5）
 Case study：DSD-Tracer on Themida
 Analyzing Themida by traditional debugger/static technique
is troublesome
 recording memory-io
 “dump” sample in static environment
19
Methodology of our study with
DSD-Tracer（#6）
 Justification for using DSD-Tracer
 Coverage of packed samples
 Low-level accuracy
 Circumventing armour techniques
 Mitigating factors in using DSD-Tracer
 No Bochs detect techniques in any sample
 4 samples/hour, 5 samples from each set of packed file
 85% of Themida samples with VM-aware techniques
20
Methodology of our study with
DSD-Tracer（#7）
 Proof of concept experiment for DSD-Tracer on VMware
 Cross-verified multiple dynamic analysis
 Implemented on VMware Workstation 6
 Invisible breakpoint
 GDB script for printing the assembly execution trace in user
mode
21
Results（#1）
 VM detection in packers
 193 different packers, 400 packed samples
 Overall VM detection rate is 1.15%
 Themida accounting for 1.03%
 ExeCryptor accounting for 0.15%
 EncPk：custom packers
22
Results（#2）
 VM detection in malware families
 Static analysis rules – disassembly
 Dynamic analysis rules – Sophos virus engine emulation
 2 million known malicious files
 A large set of knows clean files
 VM-aware samples < 1%
 Method breakdown（Table 1.）
 Family breakdown （Table 2.）
 Dial/FlashL
23
Results（#3）
24
Results（#4）
 VMWare backdoor detection method  50% VPC illegal
instruction detection method
 VPC illegal instruction detection method  93%
VMWare backdoor detection method
25
Results（#5）
 Fig. 7 VMWare backdoor detection in 2007
26
Results（#6）
 Fig. 8 VPC backdoor detections in 2007
27
Conclusion
 Combination of dynamic and static analysis is better
 2.13% VM-aware samples
28
Q&A
29
Appendix
 VMWare backdoor I/O port
 On the Cutting Edge:Thwarting Virtual MachineDetection
 Trapping worm in a virtual net
 VM、Virtual PC、Bochs比較
 http://hi.baidu.com/%CC%FA%D0%AC%B9%C3%C4%E
F/blog/item/085cc609b215f3226b60fba5.html 大陸版
 http://www.osnews.com/story/1054 國外版
30
Thanks ~
31