Regshot: This tool, as it names says, takes a snapshot of the regisrty. It basically gives you a baseline of what the registry looks like at that point in time. Given that baseline, you can then execute the suspicious executable, then take another registry snapshot. You are then able to compare both snapshots using regshot’s compare feature to find out what keys have been added, modified or deleted. It has the option of outputing its results in a text file or a nicely formated HTML file.
Regmon: Like regshot, regmon is registry utility that operates in a slightly different manner. It has the ability to give real time analysis of what keys (and their location) currently running processes are accessing. It lets you know whether the process is querying information, creating new keys, setting values, etc. Just before you execute the malware, you can have regmon running in the background capturing its information. When the program has been executed, you can stop regmon’s capture and perform your analysis. You would notice that while regmon was capturing data, it not only captured information for the malware process you are investigation, but also other processes as well that were recently accessing the registry. Thankfully, there is a nice filter feature that allows you to filter the captured data based on the process name. Although the filter is very limited, it is still beneficial to have. You can also look into another tool called procmon, who is the current successor of the tools regmon and filemon. It has the same capabilities of regmon and many more options. However, regmon still has its place and is simple to use and learn.
Filemon: This tool works in a similar fascion to regmon, but with files. It monitors processes that access files on the disk and log their actions(read, write, query, delete,etc) and whether they were successful or not. Like regmon, just before you execute the malware, you can have filemon running in the background capturing its information. When the target program has been executed, you can stop filemon’s capture and perform your analysis.
Wireshark and Netcat: It is known that some malware tend to want to replicate themselves over the network. Some may try to covertly download software or try to log onto some IRC channel to query its commands (google: botnet). These tools are coded to work covertly, so while you’re sitting at your desktop, you would not see any indication that anything is going on. Wireshark can help us understand the why, where, what, when and who questions. Why is the malware connecting out to port 4444; where is the malware trying to connect to; when or at what intervals does the malware initiate any type of network traffic; what is the malware trying to do or accomplish; who is involved (source IPs, mac addresses, domains etc.). Netcat can be set up to intercept this traffic in a proxy mode and also be used to interact/respond to services and requests.
Netstat and tasklist: Before analyzing any piece of malware, having a baseline is very vital. You are gonna need to have an idea of what the system looked like before and after the malware was run. Running netstat and tasklist before running the executable can give us a baseline of what network sessions are open and ports that are listening etc. while the tasklist command utility can give us a list of currently running processes. Tools that you can be use as well are sysinternal’s process explorer and tcpview.
Debugger, Ollydbg: To really get in depth with exactly what the executable is doing, you will have to use a debbuger to step through the system opcodes and system calls. Using a debugger is not easy for most and can take a little bit of getting use to. However, to be good at malware analysis, you cannont escape not learning how to use a debugger like IDA pro or in my case, Ollydbg.
Virtual environment: To avoid potentially infecting your main system and possibly breaking your Windows OS, you will definitly want to perform most, if not all, your analysis in a virtual environment. Virtual machines also provides us with a mechanism to roll back a host to a snapshot of a system at an earlier time. This allows us to restore the state of a system to a point just before an event occured (say the malware caused the OS to no longer start up) withing minutes. There are quite a few options avaialble for virtualization but i myself use the Virtualbox technology. Remeber to check out the system requirements of these technologies before installing to your old pentium three laptop with 256 ram.
PE tools: Sometimes malware may be packed by common packer tools, like UPX. The benifit of using a packer on an EXE file is that it can allow for the compression of the executable. However, by doing so, the original exe’s form is thus changed. Eventually, what you get is an exe within an exe. The outer layer exe will be the packers decompression code that decompresses the internal exe in memory and then executes it. The additional benifit of this is that it can make debugging of this packed executable a pain in the but. In order to properly debug the functionality of the packed executable, it must first be decompressed in order to be analyzed. Tools like PeID can help us identify a packed executable’s packer. By knowing this, we can potentially in some cases use the same packer to unpack the executable back to its original form. Another PE tool that i use is Lordpe, which allows for the modifying of the PE headers of binary executables.
These are just some tools that can be utilized in malware analysis process. I encourage you to do your own research and look up the malware analysis articles in the hakin9.org website. The articles are available in PDF format and is a little bit difficult to directly link to