Saving LSASS From W̶i̶n̶d̶o̶w̶s̶ D̶e̶f̶e̶n̶d̶e̶r̶ ?

This post is about the LSASS dumping tool that I wrote: MultiDump, for more details, check the previous post. Thanks to ballro for their help in the investigation and connecting the dots.

Update: I’ll admit that the original title is a bit of a clickbait (as if it isn’t to being with :), after testing MultiDump against a bunch of different hosts, I can definitely say that Windows Defender is not the culprit in causing LSASS to be suspended. I have seen this happen on hosts where it’s disabled. So what’s causing it? I really don’t know. There’s no code to interrupt comsvc.dll or ProcDump, so they should resume LSASS as a part of the normal routine. This issue seems to only happen on specific builds of Windows as discussed later. But the good news is that, with the auto resume feature, running MultiDump has not caused problems on any of the hosts I’ve tested.

The issue began following a Windows 10 update to version 22H2 (19045). After this update, every time I ran MultiDump, Windows would completely freeze. Initially, I thought the update might have caused issues with Visual Studio, or perhaps there was a bug in my code that only manifested in this new version of Windows. However, extensive testing found a simpler solution: turning off Windows Defender resolved the problem.

Identifying Windows Defender as the culprit didn’t immediately clarify the underlying issue. Why did Windows freeze entirely? Why were there no logs from Defender after rebooting? And why did the same setup work on Windows 11 version 23H2 (22631) but not on this specific release? The problem was found when using Process Explorer during a MultiDump session: LSASS was being inexplicably suspended:

Interestingly, using “Suspend” on the process in Process Explorer brings it back to life:

After LSASS was resumed, 3 alerts popped up from defender, the first is expected since the dump file has to be processed immediately, or it will get detected:

The other two is more concerning , it appears that Defender is somehow detecting the dump, more specifically, the child rundell32.exe process calling comsvc.dll to dump LSASS: (using ProcDump will also cause LSASS to be suspended, but no alerts somehow)

So that’s it right? Game over.

Well… If we run MultiDump again, it dumps LSASS like nothing had happened, without Defender doing a thing.

The reason Windows freezes seems clearer now, even if some details are still a bit murky. Here’s what happens: when using ProcDump or comsvc.dll to make a memory dump, the process being dumped, like LSASS, has to be paused. This allows the dump to be created correctly. However, if Windows Defender spots and stops the dumping process while LSASS is still paused, LSASS doesn’t get resumed by ProcDump or comsvc.dll. Since LSASS is crucial for Windows to work properly, leaving it suspended makes the whole system freeze up.

Since we now know that LSASS can be resumed manually, it can also be done by MultiDump. First, we will need to determine if LSASS is in fact suspended, there are no WinAPIs we can simply use, so, we will go back to our old friend NtQuerySystemInformation.

From ProcessHacker, we can find that NtQuerySystemInformation return the following structure in an array:

typedef struct _SYSTEM_PROCESS_INFORMATION
{
    ULONG NextEntryOffset;
    ULONG NumberOfThreads; // Size of the Threads member
    LARGE_INTEGER WorkingSetPrivateSize; 
    ULONG HardFaultCount; 
    ULONG NumberOfThreadsHighWatermark; 
    ULONGLONG CycleTime; 
    LARGE_INTEGER CreateTime;
    LARGE_INTEGER UserTime;
    LARGE_INTEGER KernelTime;
    UNICODE_STRING ImageName;
    KPRIORITY BasePriority;
    HANDLE UniqueProcessId;
    HANDLE InheritedFromUniqueProcessId;
    ULONG HandleCount;
    ULONG SessionId;
    ULONG_PTR UniqueProcessKey; 
    SIZE_T PeakVirtualSize;
    SIZE_T VirtualSize;
    ULONG PageFaultCount;
    SIZE_T PeakWorkingSetSize;
    SIZE_T WorkingSetSize;
    SIZE_T QuotaPeakPagedPoolUsage;
    SIZE_T QuotaPagedPoolUsage;
    SIZE_T QuotaPeakNonPagedPoolUsage;
    SIZE_T QuotaNonPagedPoolUsage;
    SIZE_T PagefileUsage;
    SIZE_T PeakPagefileUsage;
    SIZE_T PrivatePageCount;
    LARGE_INTEGER ReadOperationCount;
    LARGE_INTEGER WriteOperationCount;
    LARGE_INTEGER OtherOperationCount;
    LARGE_INTEGER ReadTransferCount;
    LARGE_INTEGER WriteTransferCount;
    LARGE_INTEGER OtherTransferCount;
    SYSTEM_THREAD_INFORMATION Threads[1]; // Threads member
} SYSTEM_PROCESS_INFORMATION, *PSYSTEM_PROCESS_INFORMATION;

Specifically, we’re interested in the SYSTEM_THREAD_INFORMATION member structure as shown here:

typedef struct _SYSTEM_THREAD_INFORMATION
{
    LARGE_INTEGER KernelTime;
    LARGE_INTEGER UserTime;
    LARGE_INTEGER CreateTime;
    ULONG WaitTime;
    PVOID StartAddress;
    CLIENT_ID ClientId;
    KPRIORITY Priority;
    KPRIORITY BasePriority;
    ULONG ContextSwitches;
    KTHREAD_STATE ThreadState;
    KWAIT_REASON WaitReason;
} SYSTEM_THREAD_INFORMATION, *PSYSTEM_THREAD_INFORMATION;

We will need 3 pieces of information from this, the thread ID UniqueThread in CLIENT_ID:

typedef struct _CLIENT_ID
{
    HANDLE UniqueProcess;
    HANDLE UniqueThread;
} CLIENT_ID, *PCLIENT_ID;

We will need the ID so that it can be resumed.

We need the ThreadState as shown here, specially, the value should be 5: Wait, as explained by Microsoft:

A state that indicates the thread is not ready to use the processor because it is waiting for a peripheral operation to complete or a resource to become free. When the thread is ready, it will be rescheduled.

Lastly, we will need the WaitReason as shown here, specially, the value should also be 5, which represents the thread is suspended.

So the logic is as follows: get an array of SYSTEM_PROCESS_INFORMATION from NtQuerySystemInformation, each representing a process. Iterate through the array and find the process we want, in this case, lsass.exe. We then iterate through the threads, locate threads with ThreadState and WaitReason being 5, and return the UniqueThread in an array:

DWORD* GetRemoteProcessSuspendedThreads(IN LPCWSTR szProcName, OUT DWORD* threadCount) {

	fnNtQuerySystemInformation		pNtQuerySystemInformation = NULL;
	ULONG							uReturnLen1 = NULL,
									uReturnLen2 = NULL;
	PSYSTEM_PROCESS_INFORMATION		SystemProcInfo = NULL,
									pOriginalSystemProcInfo = NULL;
	PSYSTEM_THREAD_INFORMATION      SystemThreadInfo = NULL;
	PVOID							pValueToFree = NULL;
	NTSTATUS						STATUS = NULL;
	*threadCount = 0;

	DWORD* suspendedThreadIds = (DWORD*)malloc(MAX_THREADS * sizeof(DWORD));

	// Fetching NtQuerySystemInformation's address from ntdll.dll
	pNtQuerySystemInformation = (fnNtQuerySystemInformation)GetProcAddress(GetModuleHandle(L"NTDLL.DLL"), "NtQuerySystemInformation");
	if (pNtQuerySystemInformation == NULL) {
		goto _EndOfFunc;
	}

	// First NtQuerySystemInformation call - retrieve the size of the return buffer (uReturnLen1)
	if ((STATUS = pNtQuerySystemInformation(SystemProcessInformation, NULL, NULL, &uReturnLen1)) != STATUS_SUCCESS && STATUS != STATUS_INFO_LENGTH_MISMATCH) {
		goto _EndOfFunc;
	}
	
	// Call NtQuerySystemInformation in a loop, since the buffer size needed can change if more processes are created
	while (TRUE) {
		SystemProcInfo = (PSYSTEM_PROCESS_INFORMATION)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, uReturnLen1);
		if (SystemProcInfo == NULL) {
			printf("[!] HeapAlloc Failed With Error : %d\n", GetLastError());
			return NULL;
		}

		STATUS = pNtQuerySystemInformation(SystemProcessInformation, SystemProcInfo, uReturnLen1, &uReturnLen2);
		if (STATUS == STATUS_SUCCESS) {
			break; // Success, break from the loop
		}
		else if (STATUS == STATUS_INFO_LENGTH_MISMATCH) {
			// Buffer was too small, free it and try again with a larger size
			HeapFree(GetProcessHeap(), 0, SystemProcInfo);
			uReturnLen1 = uReturnLen2; // Use the returned size for the next attempt
		}
		else {
			HeapFree(GetProcessHeap(), 0, SystemProcInfo);
			goto _EndOfFunc;
		}
	}
	pOriginalSystemProcInfo = SystemProcInfo;  // Keep original pointer

	// Enumerating SystemProcInfo, looking for process "szProcName"
	while (TRUE) {

		// Searching for the process name
		if (SystemProcInfo->ImageName.Length && wcsncmp(SystemProcInfo->ImageName.Buffer, szProcName, SystemProcInfo->ImageName.Length / sizeof(WCHAR)) == 0) {
			// Enumerate threads of the found process
			for (ULONG i = 0; i < SystemProcInfo->NumberOfThreads; i++) {
				PSYSTEM_THREAD_INFORMATION SystemThreadInfo = &SystemProcInfo->Threads[i];
				if (SystemThreadInfo->ThreadState == 5 && SystemThreadInfo->WaitReason == 5) { // Both ThreadState and WaitReason are 5
					if (*threadCount < MAX_THREADS) {
						suspendedThreadIds[*threadCount] = (DWORD)SystemThreadInfo->ClientId.UniqueThread;
						(*threadCount)++;
					}
				}
			}
			break;
		}

		// If we reached the end of the SYSTEM_PROCESS_INFORMATION structure
		if (!SystemProcInfo->NextEntryOffset)
			break;

		// Calculate the next SYSTEM_PROCESS_INFORMATION element in the array
		SystemProcInfo = (PSYSTEM_PROCESS_INFORMATION)((ULONG_PTR)SystemProcInfo + SystemProcInfo->NextEntryOffset);
	}

	free(pOriginalSystemProcInfo); // Free the buffer allocated for system process information

	if (*threadCount == 0) { // No matching threads found, clean up
		goto _EndOfFunc;
	}

	return suspendedThreadIds; // Return the array of matching thread IDs

// Free the SYSTEM_PROCESS_INFORMATION structure
_EndOfFunc:
	free(suspendedThreadIds);
	if (pValueToFree)
		HeapFree(GetProcessHeap(), 0, pValueToFree);
	return NULL;
}

Now, with the array of suspended thread IDs returned, we can iterate through the array and use the ResumeThread WinAPI call on the IDs to resume them.

With these functions implemented, LSASS gets resumed if its suspended, and ensuring that Windows continue to work, at this point, the program exits and Defender throws the same 3 alerts as shown earlier.

Here is perhaps the strangest part: a retry logic is implemented, if dump failure is detected, it resumes LSASS, and attempts to dump it again. MultiDump fails on the first try, and succeeds on the second, but without a single alert from Defender. And all subsequent attempts are successful, even after restarts.

Since LSASS got suspended the first time, it make sense that it was detected, but why is there no alerts? And more importantly, why does Defender only detect it once, and never again?

I don’t have a definitive answer, but my best guess would be on adaptive heuristic detection mechanisms. Since the process attempting to dump LSASS was terminated before any actual dump file could be created, and assuming LSASS was quickly resumed afterwards - Defender might not flag the activity as malicious. Consequently, it could mark similar subsequent operations as safe, and adjusting its criteria based on the initial outcome.

This was tested on Windows 10 22H2 (19045), since this behaviour only happens once, I have to revert to a snapshot to test, it has the following Windows Defender version:

AMEngineVersion                  : 1.1.23110.2
AMProductVersion                 : 4.18.23110.3
AntivirusSignatureVersion        : 1.403.3375.0
AntispywareSignatureVersion      : 1.403.3375.0

I also have a Windows 11 23H2 (22631) VM that MultiDump runs flawlessly on, without the above-mentioned behaviour and a later Windows Defender version:

AMEngineVersion                  : 1.1.23110.2
AMProductVersion                 : 4.18.23110.3
AntivirusSignatureVersion        : 1.403.3385.0
AntispywareSignatureVersion      : 1.403.3385.0

So it seems that only this specific version of Windows causes issues. I will update this post if I find more information.