Malware Analysis: Viruses, Ransomware, and Trojans

Cybersecurity Cybersecurity 8 min read 1627 words Beginner ExcellentWiki Editorial Team

Malware analysis is the art and science of understanding malicious software. Security analysts dissect malware to determine what it does, how it operates, who created it, and how to defend against it. This knowledge drives detection signatures, containment strategies, and threat intelligence across the security community. Without malware analysis, defenders are blind — responding to symptoms without understanding the underlying threat.

Types of Malware

Viruses

Viruses attach themselves to legitimate programs and spread when the host program executes. They require human action to propagate — opening an infected attachment, running an infected installer, or mounting infected media. File infector viruses modify executable files by appending or prepending their code. Macro viruses embed malicious code in document scripts (Microsoft Office VBA macros).

Worms

Worms spread autonomously across networks without requiring human interaction. They exploit network services, replicate themselves, and scan for new targets. Worms are the fastest-spreading malware type because they need no user action. The Morris worm of 1988 brought down 10% of the early internet. WannaCry (2017) used the EternalBlue exploit to spread across 150 countries in a single day.

Trojans

Trojans disguise themselves as legitimate software while performing malicious functions. A downloaded game might install a backdoor. A PDF viewer might steal browser credentials. Unlike viruses and worms, Trojans do not self-replicate — they rely on social engineering for delivery. Emotet started as a banking Trojan and evolved into a malware delivery platform used by ransomware gangs worldwide.

Ransomware

Ransomware encrypts files and demands payment for decryption. Modern ransomware combines worm-like propagation with sophisticated encryption. Ryuk targeted enterprise networks with manual deployment. Maze added data theft before encryption, pressuring victims with the threat of data leaks. LockBit and BlackCat (ALPHV) operate as ransomware-as-a-service, with developers leasing their malware to affiliates.

Rootkits

Rootkits hide their presence by subverting operating system mechanisms. They intercept system calls, hide processes and files, and maintain persistent access. Kernel-mode rootkits operate at the same privilege level as the operating system, making them extremely difficult to detect. User-mode rootkits hook application APIs or DLL imports, hiding from user-level tools.

Static Analysis

Static analysis examines malware without executing it. The malware never runs, making this approach safe but limited — dynamic behavior cannot be observed.

File Fingerprinting

Calculate cryptographic hashes of the malware sample: MD5, SHA-1, SHA-256. Hashes identify known samples in threat intelligence databases like VirusTotal. A hash match immediately provides detection rates, behavior reports, and community notes. Hash-based detection is trivial for attackers to bypass — modifying a single byte changes the hash.

String Analysis

Extract embedded strings from the binary. Strings may reveal:

IP addresses and domain names (C2 servers)
Registry keys and file paths (persistence locations)
API function names (capabilities)
Error messages (debug information)
Encryption keys and initialization vectors

strings malware.exe | sort -u
strings -n 8 malware.exe | grep -E 'https?://'

Packed binaries have few readable strings — high entropy and short string length indicate packing or obfuscation.

PE Analysis

Portable Executable (PE) files (Windows executables, DLLs, SYS drivers) have a structured format that reveals significant information. Tools: pefile (Python), PE-bear, CFF Explorer.

Key PE header fields:

Entry point — where execution begins. Suspicious if in a non-standard section.
Imported functions — DLLs and APIs the binary calls. URLDownloadToFile suggests downloader capability. CreateRemoteThread suggests process injection. RegSetValueEx suggests persistence.
Section names — non-standard names (.text, .data, .rdata are standard; random names suggest packing).
Compilation timestamps — often falsified, but anomalies indicate tampering.

Packer Detection

Malware authors pack or obfuscate code to evade signature-based detection. Packed binaries have:

High entropy (compressed or encrypted content)
Few readable strings
Small number of imports (the unpacker stub, not the actual code)
Suspicious section names (UPX0, UPX1 for UPX packing)

Detect packers with PEiD, Detect It Easy (DiE), or Exeinfo PE. Unpacking requires manual effort — run the malware in a debugger and dump memory after the unpacking stub executes.

Dynamic Analysis

Dynamic analysis executes malware in a controlled environment to observe behavior. This provides richer information but requires careful isolation.

Sandboxing

Automated sandboxes execute malware and report behavior. Cuckoo Sandbox (open-source), Joe Sandbox, Any.Run, and Hybrid Analysis capture:

File system changes (files created, modified, deleted)
Registry modifications (persistence mechanisms)
Network connections (C2 domains, IPs, protocols)
Process creations and injections
API call traces

Sandbox reports provide rapid initial analysis. However, malware increasingly detects sandbox environments:

Checks system uptime (just-created VM has low uptime)
Looks for analysis tools (process names, window titles, drivers)
Checks MAC address prefixes (VMware, VirtualBox, QEMU)
Examines hardware characteristics (RAM size, disk size, CPU cores)

Sandbox-aware malware behaves innocuously when it detects analysis and activates only on real systems.

Network Analysis

Monitor network traffic during execution. Tools: Wireshark, tcpdump, Fakenet-NG, INetSim.

Signals to look for:

DNS queries to algorithmically generated domains (DGAs) — random-looking subdomains
HTTP/HTTPS connections to unusual IPs or domains
Non-standard protocols on standard ports
Encrypted C2 traffic (HTTPS with custom certificates)
Data exfiltration patterns (base64-encoded strings in HTTP requests)

INetSim simulates network services, so malware’s DNS queries resolve to localhost where fake services respond. This triggers C2 communication without contacting actual attacker infrastructure.

Process and Memory Analysis

Tools like Process Monitor (Procmon), Process Explorer, and Process Hacker capture real-time process activity:

Process creation — unexpected process spawning (Word launching PowerShell)
Process injection — WriteProcessMemory + CreateRemoteThread pattern
Service installation — new Windows services
Scheduled task creation
WMI event subscription persistence

Memory Analysis

When malware executes, it leaves artifacts in RAM. Memory forensics tools like Volatility analyze RAM dumps:

# Identify the operating system and profile
volatility -f memory.dump imageinfo

# List running processes
volatility -f memory.dump pslist

# Extract injected code
volatility -f memory.dump malfind

# Dump network connections
volatility -f memory.dump netscan

Memory analysis reveals:

Hidden processes (rootkit techniques)
Injected code in legitimate process memory
Unpacked malware in memory (after unpacking stub runs)
Network connections not visible from network monitoring

Reverse Engineering

Reverse engineering disassembles malware to understand its logic at the code level. This is the most time-intensive analysis phase.

Disassemblers convert machine code to assembly language. IDA Pro (commercial) and Ghidra (open-source, developed by NSA) are the standard tools. Ghidra’s decompiler produces C-like pseudo-code, making it more accessible than reading raw assembly.

Debuggers (x64dbg, WinDbg, OllyDbg) execute malware step-by-step, allowing analysis of:

Branching logic and conditional behavior
Encryption routines (key derivation, decryption loops)
Anti-analysis techniques (timing checks, debugger detection)
C2 protocol implementation

Reverse engineering answers the questions that static and dynamic analysis cannot: How does the encryption algorithm work? Can files be decrypted without paying the ransom? What is the full set of capabilities in the payload?

Defense Strategies

Signature-Based Detection

Antivirus signatures match known malware patterns. YARA rules provide flexible pattern matching for file characteristics and behavior:

rule RansomwareFileExt
{
  strings:
    $encrypt_ext = ".encrypted" nocase
    $ransom_note = "README_TO_DECRYPT" nocase
  condition:
    all of them
---

YARA rules are shared through open-source repositories (YARA Forge, Valhalla) and commercial threat intelligence feeds.

Behavioral Detection

Behavioral detection identifies malicious activity patterns regardless of file signature. EDR solutions monitor for:

Mass file encryption operations
Suspicious process injection
Unusual network connections from business applications
Registry modification for persistence
Privilege escalation attempts

Threat Intelligence Sharing

Share malware indicators with the security community. Platforms: MISP (Malware Information Sharing Platform), ThreatConnect, CrowdStrike Falcon Intelligence. The more organizations share, the faster everyone can detect new threats. The Cyber Threat Alliance and Information Sharing and Analysis Centers (ISACs) facilitate cross-sector collaboration.

Frequently Asked Questions

What is the difference between static and dynamic analysis?

Static analysis examines malware without executing it — checking strings, PE headers, and imported functions. Dynamic analysis runs the malware in a sandbox to observe file system changes, network connections, and process behavior. Static analysis is safer (the malware never runs) but provides limited insight. Dynamic analysis reveals actual behavior but requires secure isolation. Both are necessary for comprehensive analysis.

How do malware analysts avoid infection during analysis?

Analysis occurs in isolated environments — virtual machines with no network access to production systems. Sandboxes are snapshotted to a clean state before each analysis. Network isolation tools (INetSim, Fakenet-NG) simulate internet services without actually connecting to attacker infrastructure. Forensic workstations use write-blockers to prevent any data from being written to analysis drives.

What tools do professional malware analysts use?

Professional analysts use a combination of: Ghidra or IDA Pro for reverse engineering, x64dbg for debugging, Volatility for memory forensics, Procmon for process monitoring, Wireshark for network analysis, YARA for rule creation, and Cuckoo Sandbox or Joe Sandbox for automated dynamic analysis. Linux analysts use ltrace, strace, and gdb for similar capabilities.

Can ransomware always be decrypted?

No. Only some ransomware variants have published decryption tools. Decryption may be possible if the ransomware uses a flawed encryption implementation (same key for all victims, key embedded in binary, weak random number generation). Well-implemented ransomware uses asymmetric encryption — files are encrypted with a unique key, which is encrypted with the attacker’s public key. Without the attacker’s private key, decryption is computationally infeasible. NoDecipher and ID Ransomware maintain databases of known decryptable variants.

Conclusion

Malware analysis is a continuous cat-and-mouse game. Attackers evolve their techniques — packing, obfuscation, sandbox evasion, fileless execution, living-off-the-land binaries. Analysts evolve their tools and methods correspondingly. Understanding malware is essential for building effective defenses, informing threat intelligence, and responding to security incidents. The analysis process — static, dynamic, memory, reverse engineering — provides a complete picture that no single technique can provide alone.

For a comprehensive overview, read our article on Cloud Security Architecture.

For a comprehensive overview, read our article on Cloud Security Guide.

Share this article

X LinkedIn Facebook Email