Dynamic Malware Analysis: How It Works and How It Differs from Static Analysis

A file lands on your network. Your antivirus doesn't flag it. Static signatures find nothing. Is it safe?

Dynamic malware analysis answers that question by doing something static analysis cannot: it executes the file and watches what happens. Instead of examining the file's code for known patterns, dynamic analysis runs the suspicious file in a controlled environment and observes its behavior in real time. Does it create new files? Does it modify the registry? Does it establish network connections to command-and-control servers? Does it attempt privilege escalation? Does it encrypt files?

Behavioral analysis catches what signature-based detection misses. A new malware variant with no matching signature still exhibits malicious behavior when executed: it still calls home, still drops payloads, still attempts persistence. Dynamic malware analysis is how sandboxes detect threats in real time. For a comprehensive guide to sandbox technology, see our sandbox cybersecurity guide.

This guide covers how dynamic malware analysis works, the behavioral indicators it detects, the tools used by security teams, how it differs from static analysis, when to use each approach, and how malware analysis fits into a broader security programme.

What Is Dynamic Malware Analysis?

Dynamic malware analysis is the process of executing a suspicious file or program in a controlled, isolated environment (a sandbox) and monitoring its runtime behavior to determine whether it is malicious. The analysis observes what the file does rather than what the file looks like.

What Dynamic Analysis Observes

File system activity. Does the program create, modify, or delete files? Does it drop additional executables? Does it access sensitive directories?

Registry changes. Does it create persistence mechanisms through registry keys? Does it modify system configuration?

Network activity. Does it establish outbound connections? What domains and IP addresses does it contact? Does it use DNS tunneling? Does it download additional payloads?

Process behavior. Does it spawn child processes? Does it inject code into other processes? Does it attempt to escalate privileges? Does it interact with system services?

API calls. Which system calls does it make? Calls to cryptographic APIs suggest encryption (potential ransomware). Calls to networking APIs indicate communication. Calls to process manipulation APIs indicate injection or evasion.

Memory activity. Does it allocate memory regions with executable permissions? Does it unpack code in memory (suggesting packed or obfuscated malware)?

Why Behavior Reveals What Signatures Miss

Malware authors change their code constantly to evade signature detection. A new variant of ransomware may have entirely different code from its predecessor while exhibiting identical behavior: encrypt files, drop a ransom note, establish C2 communication. The behavior is the constant. The code is the variable. Dynamic analysis targets the constant.

How Dynamic Malware Analysis Works

Step 1: Environment Setup

The analysis environment is a controlled sandbox: a virtual machine or isolated system configured to mimic a typical target environment. The sandbox looks like a normal Windows workstation with standard applications, network connectivity (real or simulated), and user activity to prevent sandbox-aware malware from detecting the analysis environment.

Environment considerations: Operating system matching the target (Windows 10/11 for most enterprise malware). Common applications installed (Office, browsers, PDF readers). Simulated network services (DNS, HTTP, SMTP) for capturing communication. Monitoring tools installed at the hypervisor level (below the guest OS) to avoid detection.

Step 2: Sample Execution

The suspicious file is executed within the sandbox. Depending on the file type, this means running an executable, opening a document in the appropriate application (Word for .docx, Adobe for .pdf), executing a script through its interpreter, or browsing to a URL.

Execution triggers: Some malware requires user interaction (clicking "Enable Macros," for example). Automated analysis environments simulate user behavior: clicking dialog boxes, typing, scrolling, and waiting. Sophisticated sandboxes simulate multiple execution scenarios.

Step 3: Behavior Monitoring

While the sample runs, monitoring tools capture every observable action.

System-level monitoring captures:

File creation, modification, and deletion with full paths and content hashes. Registry key creation, modification, and deletion. Process creation and termination with command-line arguments. Network connections with destination IPs, ports, and protocols. DNS queries with requested domains. HTTP requests and responses. API calls with parameters and return values. Screenshots at intervals showing visual behavior (ransom notes, fake alerts).

Step 4: Network Traffic Analysis

Network monitoring captures all communication the malware attempts.

What network analysis reveals: C2 server locations (IP addresses and domains). Communication protocols (HTTP, HTTPS, DNS tunneling, custom protocols). Data exfiltration attempts (volume, destination, encoding). Download of additional payloads (second-stage malware). Lateral movement attempts within the simulated network.

Step 5: Behavioral Classification

Observed behaviors are classified against known malware behavior patterns.

Behavioral indicators of common malware types:

Behavior PatternLikely ClassificationEncrypts files + drops ransom note + contacts C2RansomwareCaptures keystrokes + sends to remote serverKeylogger/InfostealerInjects into browser process + modifies banking sitesBanking trojanEstablishes persistent C2 + awaits commandsRAT (Remote Access Trojan)Encrypts and exfiltrates documentsData theftModifies boot sequence + corrupts filesystemWiperDownloads and installs additional malwareDropper/LoaderSpreads to network shares + replicatesWorm

Step 6: Report Generation

The analysis produces a behavioral report documenting every observed action, network indicator, file artifact, and behavioral classification. This report feeds threat hunting with indicators of compromise (IOCs) and behavioral signatures.

Dynamic vs Static Malware Analysis

What Static Analysis Does

Static analysis examines a file without executing it: file headers, embedded strings, imported libraries, code structure, digital signatures, entropy analysis, and pattern matching against known malware signatures.

Static analysis strengths: Fast (seconds per file). Scales to millions of samples. No execution risk. Identifies known malware families instantly. Extracts IOCs (hashes, strings, embedded URLs) without execution.

Static analysis weaknesses: Packed or obfuscated malware is opaque to static analysis. Novel malware with no matching signature passes undetected. Polymorphic malware changes its code each time, evading signature matching. Cannot observe runtime behavior.

What Dynamic Analysis Adds

Dynamic analysis reveals what the malware actually does when it runs. A heavily obfuscated file that looks benign to static analysis reveals its true nature when it executes: unpacking in memory, establishing C2 connections, dropping additional payloads, and encrypting files.

Dynamic analysis strengths: Detects novel and zero-day malware by behavior. Sees through packing, encryption, and obfuscation. Captures network IOCs (C2 infrastructure). Observes the complete attack chain.

Dynamic analysis weaknesses: Slower than static analysis. Sandbox-aware malware may detect the analysis environment and hide malicious behavior. Some malware requires specific triggers (time-based, geography-based, user interaction) that the sandbox may not provide. Limited by sandbox capacity for high-volume analysis.

The Combined Approach

Professional malware analysis uses both techniques together. Static analysis provides fast initial triage: known malware is classified immediately, suspicious files are flagged for deeper analysis. Dynamic analysis provides behavioral confirmation: flagged files are executed in sandboxes to observe actual behavior.

This layered approach mirrors the DAST vs SAST paradigm in application security: static analysis examines code, dynamic analysis tests running behavior, and the combination provides comprehensive coverage.

Dynamic Malware Analysis Tools

Sandbox Platforms

Cuckoo Sandbox. Open-source automated malware analysis system. Executes files in virtual machines, monitors behavior, captures network traffic, and generates detailed reports. The most widely deployed open-source sandbox.

Joe Sandbox. Commercial deep malware analysis platform. Supports Windows, macOS, Linux, Android, and iOS analysis. Advanced anti-evasion technology. Hypervisor-level monitoring.

ANY.RUN. Interactive cloud-based sandbox allowing real-time interaction with malware during execution. Analysts can click buttons, enter data, and observe behavior in real time rather than relying on automated interaction.

VMRay. Hypervisor-based analysis platform. Monitoring at the hypervisor level makes detection by sandbox-aware malware extremely difficult. Enterprise-focused with API integration.

Hybrid Analysis (CrowdStrike). Free community sandbox with CrowdStrike Falcon integration. Combines static and dynamic analysis in a single report.

Network Analysis Tools

Wireshark. Network protocol analyzer capturing and dissecting all network traffic generated by the malware sample during dynamic execution.

FakeNet-NG. Simulates network services (DNS, HTTP, SMTP) to capture malware communication without allowing actual internet access. Prevents C2 communication while recording what the malware attempts.

INetSim. Internet services simulator. Provides fake HTTP, HTTPS, DNS, SMTP, and FTP services for malware to interact with during analysis.

System Monitoring Tools

Process Monitor (ProcMon). Microsoft tool monitoring real-time file system, registry, and process activity. Essential for Windows malware analysis.

Regshot. Registry comparison tool. Takes snapshots before and after malware execution, identifying every registry modification.

API Monitor. Captures API calls made by applications. Reveals which system functions the malware invokes.

Memory Analysis

Volatility. Open-source memory forensics framework. Analyzes memory dumps from sandboxes to find injected code, hidden processes, and network connections invisible to live system tools.

Sandbox Evasion: How Malware Fights Back

Sophisticated malware detects analysis environments and alters its behavior to appear benign.

Common Evasion Techniques

Environment detection. Malware checks for virtual machine artifacts: VM-specific hardware identifiers, registry keys indicating hypervisor presence, sandbox-associated file paths, or analysis tool processes running.

Timing-based evasion. Malware sleeps for extended periods (hours or days) before executing its payload, exceeding typical sandbox analysis windows. Or it checks the system clock and only executes during business hours.

User interaction requirements. Malware waits for specific user actions (mouse movement, keyboard input, specific application launches) before executing. Automated sandboxes without human interaction simulation miss these triggers.

Geographic targeting. Malware checks the system's IP geolocation, language settings, or keyboard layout and only executes in targeted regions.

Hardware fingerprinting. Checks CPU core count, RAM size, disk size, and screen resolution. Sandboxes with minimal resources are detected.

Anti-Evasion Countermeasures

Modern sandboxes counter evasion through realistic hardware profiles (matching production workstations), time manipulation (accelerating sandbox clocks past sleep timers), human interaction simulation (automated mouse movement, typing, clicking), bare-metal analysis (physical machines eliminating VM artifacts), and hypervisor-level monitoring (invisible to the guest OS).

Dynamic Analysis in Incident Response

When a breach is detected, dynamic malware analysis becomes an immediate priority.

Incident Response Workflow

Step 1: Containment. Isolate affected systems. Collect malware samples from compromised endpoints.

Step 2: Dynamic analysis. Execute collected samples in sandboxes. Identify C2 infrastructure, lateral movement techniques, and persistence mechanisms.

Step 3: IOC extraction. Extract network indicators (C2 IPs, domains), file indicators (hashes, dropped files), and behavioral indicators (registry keys, scheduled tasks) from analysis results.

Step 4: Scope assessment. Search the environment for extracted IOCs to determine breach scope. Feed behavioral indicators into threat hunting hypotheses.

Step 5: Eradication. Use analysis findings to remove all malware artifacts, disable persistence mechanisms, and block C2 communication.

The ransomware threat makes dynamic analysis especially critical. Understanding how a ransomware variant behaves (encryption method, key exchange, C2 communication) directly informs whether data recovery is possible. For current ransomware threat context, see our ransomware trends analysis.

Dynamic Analysis and Supply Chain Security

Supply chain attacks deliver malware through trusted software update channels, open-source packages, or compromised development tools. Dynamic analysis of software packages before deployment into production environments can detect malicious behavior that code review alone might miss: backdoor communications, data exfiltration, or credential harvesting embedded in otherwise legitimate packages.

Integrating dynamic analysis into CTEM programmes ensures continuous evaluation of threats entering your environment through any vector.

Building a Malware Analysis Capability

For Security Operations Teams

Minimum viable capability: Deploy an automated sandbox (Cuckoo, Joe Sandbox Cloud, or ANY.RUN) that SOC analysts submit suspicious files to. Integrate sandbox analysis into alert triage workflow. When an alert involves a file, analyze it before escalating.

Intermediate capability: Automate file submission from email gateway and endpoint detection. Every flagged file is automatically analyzed. Network IOCs from analysis are automatically fed into SIEM and blocking rules.

Advanced capability: Dedicated malware analysis team with both automated and manual analysis capability. Memory forensics. Custom sandbox configurations for targeted malware. Threat intelligence production from analysis findings.

Integration with Security Testing

Dynamic malware analysis complements penetration testing and offensive security:

Red team exercises use custom malware and tooling. Dynamic analysis of red team payloads validates whether detection capabilities catch attacker tools. See our red teaming guide.

Cloud security testing evaluates whether cloud-based malware detection services (AWS GuardDuty, Azure Defender) identify threats through behavioral analysis.

Network penetration testing assesses whether network-based detection identifies C2 communication patterns.

Attack surface management identifies entry points where malware could enter the environment.

Dynamic Analysis for Compliance

Several compliance frameworks require or benefit from malware analysis capabilities.

NIST CSF. Detect function (DE.CM) requires continuous monitoring including malware detection. Dynamic analysis validates detection effectiveness. See our NIST CSF guide.

ISO 27001. Annex A.8.7 (Protection Against Malware) requires malware detection and prevention. Dynamic analysis demonstrates advanced detection beyond signature-based tools. See our ISO 27001 guide.

PCI DSS. Requirement 5 mandates anti-malware for systems in the cardholder data environment. Dynamic analysis supplements traditional antivirus. See our PCI DSS guide.

For multi-framework mapping, see our penetration testing compliance guide.

Dynamic Malware Analysis Checklist

Environment Setup

• Sandbox platform selected and deployed (Cuckoo, Joe Sandbox, ANY.RUN, VMRay)
• Analysis VMs configured with realistic OS, applications, and hardware profiles
• Network monitoring capturing all sandbox traffic (Wireshark, FakeNet-NG)
• Anti-evasion measures implemented (realistic hardware, user simulation, time manipulation)
• Snapshot/restore capability for rapid analysis reset
• Analysis environment isolated from production networks

Analysis Process

• File submitted to sandbox with appropriate execution triggers
• Execution monitored for minimum analysis duration (5+ minutes, longer for evasive samples)
• File system changes captured (created, modified, deleted files)
• Registry modifications captured
• Process activity captured (creation, injection, privilege escalation)
• Network traffic captured (DNS, HTTP, raw connections)
• API calls logged
• Memory dump collected for advanced analysis
• Screenshots captured at intervals

Post-Analysis

• Behavioral classification completed (ransomware, RAT, infostealer, etc.)
• IOCs extracted (hashes, IPs, domains, file paths, registry keys)
• IOCs fed into SIEM and blocking rules
• Report generated with behavioral summary and IOCs
• Findings correlated with threat intelligence
• Detection gaps identified and remediation planned

How AppSecure Relates to Malware Analysis

AppSecure's offensive security testing and red teaming services test whether your malware detection capabilities work under realistic conditions.

Detection Validation. Red team exercises test whether your sandboxes, EDR, and SIEM detect custom tooling that mimics real-world malware behavior. If your dynamic analysis platform can't detect red team payloads, it won't detect real adversary tools.

Comprehensive Security Testing. Penetration testing across web applications, APIs, cloud infrastructure, and networks identifies the vulnerabilities that malware would exploit. Fix vulnerabilities through testing. Detect exploitation through analysis.

Continuous penetration testing and PTaaS maintain ongoing security validation. Application security assessment provides comprehensive coverage.

Contact AppSecure:

• Schedule Security Testing
• Red Teaming Services
• Offensive Security Testing

Frequently Asked Questions

1. What is dynamic malware analysis?

Dynamic malware analysis is the process of executing a suspicious file in a controlled, isolated environment (sandbox) and monitoring its runtime behavior to determine whether it is malicious. The analysis observes file system changes, registry modifications, network communications, process behavior, API calls, and memory activity. Unlike static analysis that examines code without executing it, dynamic analysis reveals what the malware actually does, catching novel threats that evade signature-based detection.

2. How does dynamic analysis differ from static analysis?

Static analysis examines a file's code, structure, strings, and signatures without executing it. It is fast, scalable, and risk-free but cannot see through obfuscation or detect novel malware without matching signatures. Dynamic analysis executes the file and observes behavior, seeing through packing and obfuscation but requiring more time and sandbox resources. Static analysis answers "what does this code look like?" Dynamic analysis answers "what does this code do?" Professional analysis uses both together.

3. What behavioral indicators does dynamic analysis detect?

Dynamic analysis detects file system activity (creating or encrypting files), registry modifications (persistence mechanisms), network connections (C2 communication, data exfiltration), process behavior (code injection, privilege escalation), API calls (cryptographic operations suggesting ransomware, networking operations suggesting communication), and memory activity (unpacking obfuscated code). These behavioral indicators reveal malware type and capability regardless of how the code is written or obfuscated.

4. What tools are used for dynamic malware analysis?

Sandbox platforms: Cuckoo Sandbox (open-source), Joe Sandbox, ANY.RUN (interactive), VMRay (hypervisor-based), Hybrid Analysis. Network analysis: Wireshark, FakeNet-NG, INetSim. System monitoring: Process Monitor, Regshot, API Monitor. Memory analysis: Volatility framework. Organizations typically deploy an automated sandbox platform for SOC analyst use, with advanced tools for dedicated malware analysis teams.

5. How does malware evade dynamic analysis?

Sophisticated malware detects sandbox environments through VM artifact detection (hypervisor-specific hardware, registry keys), timing delays (sleeping for hours before executing), user interaction requirements (waiting for mouse movement or specific actions), geographic targeting (checking IP location or language settings), and hardware fingerprinting (detecting minimal CPU, RAM, or disk indicating a sandbox). Modern sandboxes counter with realistic hardware profiles, time acceleration, user simulation, and hypervisor-level monitoring.

6. How does dynamic analysis relate to sandboxing?

Dynamic malware analysis is the core technique inside every sandbox. A sandbox provides the isolated execution environment. Dynamic analysis provides the monitoring and behavioral observation. When a sandbox "detects a threat," it is dynamic analysis that identified the malicious behavior. The sandbox is the container; dynamic analysis is the detection methodology. For a complete guide to sandbox technology, see our sandbox cybersecurity guide.

7. When should security teams use dynamic analysis?

When investigating suspicious files that pass static signature detection. During incident response to understand malware capabilities, extract IOCs, and determine breach scope. When evaluating software packages for supply chain threats. When threat hunting based on behavioral hypotheses. When validating that detection tools (EDR, SIEM, sandboxes) correctly identify malicious behavior. Automated sandbox analysis should be integrated into SOC alert triage workflows.

8. Can dynamic analysis detect zero-day malware?

Yes. Zero-day malware has no matching signature for static detection but still exhibits malicious behavior when executed: establishing C2 connections, modifying system configurations, encrypting files, or exfiltrating data. Dynamic analysis detects these behaviors regardless of whether the specific malware variant has been seen before. This is the primary advantage of behavioral over signature-based detection.

9. How does dynamic analysis fit into incident response?

During incident response, dynamic analysis of collected malware samples reveals C2 infrastructure (enabling network blocking), persistence mechanisms (enabling removal), lateral movement techniques (enabling scope assessment), and data exfiltration methods (enabling impact assessment). IOCs extracted from analysis feed SIEM hunting queries and blocking rules. Behavioral understanding informs whether data recovery is possible (critical for ransomware incidents).

10. What is the relationship between malware analysis and penetration testing?

Penetration testing identifies vulnerabilities that malware would exploit. Malware analysis evaluates whether detection catches exploitation. Red team exercises use custom tooling that mimics real malware, testing whether dynamic analysis platforms detect adversary tools. Together, penetration testing and malware analysis capabilities address both prevention (fix vulnerabilities) and detection (identify exploitation attempts).