Blog:

Hooking .NET Framework executables with Detours under Wine

This article covers a brief history of the .NET Framework, looking at how the PE files are structured and loaded under both Windows and Wine. We introduce a Wine patch that enables Detours to correctly hook .NET Framework binaries.

Contents

Introduction

Hooking into system APIs allows the user to detect when a function of interest is called, and run custom code when it is. This allows access to low-level information about the behaviour and execution of the API call being hooked. Depending on the use case, we can retrieve information such as input variables and return values, we can modify the API’s behaviour during execution, and many other possibilities. Another common use case is performance analysis, providing insight into the application speed and execution to developers.

During our time migrating Windows workloads to Wine we often need further tooling, outside of normal debugging, to identify the differences in application behaviour when running under Windows and Wine. System API hooking enables us to instrument this behaviour and identify how this may differ across both platforms.

Microsoft Detours is a popular hooking library provided by Microsoft themselves. Detours enables the user to hook Windows system APIs in a way that is consistent across the different versions of Microsoft Windows, as well as Wine. Given that Detours functions the same way on both Windows and Wine, it is an excellent tool when comparing the behaviour of applications running on both Windows and Wine.

We have used Detours consistently when investigating application incompatibilities for both native and .NET applications with consistent success. However, we have recently discovered that .NET Framework applications are not being hooked correctly by Detours while running on Wine.

Most .NET applications in active development will be using “modern” .NET, i.e. version 5.0 and newer. Older versions of .NET are referred to as .NET Framework applications, and many of these older tools are still in use today. In order to understand why Detours is not hooking .NET Framework applications, we must understand how the executable structures differ from those used in modern .NET.

During this blog post we will be diving into the details on some of the history surrounding the .NET Framework and how it behaves while running under both Windows and Wine. We will identify some of the unique characteristics of .NET Framework applications, and how the underlying operating system loads them. We will also discuss the issues we encountered during this investigation, how we have identified the differences in Windows and Wine loading behaviours, and introduce a Wine patch that enables us to use Microsoft Detours to hook .NET Framework applications.

Technology

.NET Framework

The .NET Framework is at the heart of our investigation, and in order to understand why modern .NET applications can be hooked under Wine by Detours while older versions cannot, we must look into how the technology has matured and developed over its lifetime. The first version of the .NET Framework was released in February 2002, just 6 months after the release of Windows XP. Microsoft needed to ensure that their new programming framework would support older operating systems, and so the early versions of the .NET Framework supported Windows NT 4.0, as well as Windows 98 and higher.

Before diving into this it is important to clear up some terminology. Depending on where you look the term “.NET” can refer to one of two things - either the older versions of what is officially called the .NET Framework, or the newer cross-platform versions which were originally called .NET Core, but are now simply called .NET. In this post we will always use the term .NET Framework to refer to the older, Windows-only versions.

The .NET Framework allows source code to be compiled to Microsoft Intermediate Language (MSIL), now known as Common Intermediate Language (CIL), designed to be able to run on any system which has a compatible runtime. This intermediate code is then translated by the Common Language Runtime (CLR) to native machine code which can run on the host system (known as Just-In-Time compilation, or JIT compilation). Any code whose execution is managed by a runtime, such as the CLR, is known as managed code.

Windows XP included support for both 32-bit architecture (x86) and Itanium 64-bit architecture (IA-64), but .NET Framework 1.0 did not natively support 64-bit processing. In April 2005 Microsoft released Windows XP Professional x64, providing official support for AMD 64-bit architecture (x64). Later that year .NET Framework 2.0 was released, now with 64-bit support. One of the .NET Framework’s key design features was portability, and in order to support this a new target architecture was added - “AnyCPU”. AnyCPU code would run as a 64-bit process on 64-bit architecture, and as 32-bit on 32-bit architecture. Since AnyCPU managed code is also typesafe, the JIT compiler will determine the assembly types needed to run on the specific host architecture.

A change in how .NET Framework applications were compiled occurred with the transition from .NET Framework 4.0 to 4.5 (in 2012). Running the .NET Framework application as 64-bit on 64-bit hardware seemed like the best option, but it led to unintended side effects due to applications inadvertently attempting to load bundled 32-bit dependencies on 64-bit machines. To resolve this problem Microsoft introduced a compiler flag for the .NET Framework that configured the value in the .NET Header flag COMIMAGE_FLAGS_32BITPREFERRED when compiled with AnyCPU as the target. As of .NET Framework 4.5 this flag is turned on by default, and must be turned off by the developer. AnyCPU was intended to make development of .NET applications easier, but there are still questions about whether it adds value or not. This will prove important further along into our investigation, but the behavioural differences can be found in a post that outlines how .NET Framework 4.0 AnyCPU binaries behave:

System Architecture Application Architecture Process IL Code
x86 x86 x86 x86
AnyCPU x86 x86
x64 x86 x86 (WoW64) x86
x64 x64 x64
AnyCPU (32BITPREFERRED) x86 (WoW64) x86
AnyCPU x64 x64
Table 1: Process and code bitness for different .NET Framework build configurations under 32-bit and 64-bit host platforms.

In 2016 Microsoft released .NET Core 1.0, an open-source, cross-platform alternative to the .NET Framework, and in 2020 (with the release of .NET 5.0) the “Core” tag was dropped altogether. In order to minimise confusion there was never a .NET Core 4 release, and all updates to the .NET Framework have kept to the 4.x version tags. When we talk about “Modern .NET” we are talking about .NET 5 and higher. In modern .NET when you compile an application with the default build configuration, you end up with both an EXE file and a DLL. The EXE file is a native application and is also known as the apphost, it is what is used to instantiate the CLR with the correct runtime and SDK. This change was initially added in NET Core 3.0.

As discussed, this blog post focusses on the behaviour of .NET Framework 1.0 through to 4.8 compiled executables and the behaviour that these binaries exhibit when running on both Windows and Wine.

Mono

The .NET Framework was a Windows-only development platform, and in 2001 an open-source alternative known as Mono was announced and released. The goal of the Mono project was to support the development and execution of .NET Framework binaries on multiple platforms, including Linux, macOS, BSD and Windows. It was developed based on the ECMA Standards released by Microsoft. Mono had several homes over the years, including Novell, Xamarin and Microsoft. Eventually, in 2024 Mono was donated to the Wine project by Microsoft, after around 5 years with no major releases, and Wine Mono is now the standard way to run .NET Framework applications under Wine. A core advantage to using Wine Mono for the purposes of our investigation is that it is open source, allowing us to view the code that loads the images (executable files on disk) into memory. However, Wine Mono is not the .NET Framework, and the issue we are most interested in investigating is clearly an extreme edge case. We cannot ignore the possibility that the problem lies within an implementation detail of Mono, and not in either Wine or the .NET Framework. If our initial investigation into how .NET Framework binaries are loaded under Wine does not yield any clear solutions then we will need to investigate differences between the .NET Framework and Mono loading process.

Binary structure

To understand the structure of .NET Framework binaries, also known as Portable Executable (PE) files, we first need to identify how they are constructed and what differences exist between different versions of the .NET Framework. Using PE-Bear, a popular PE exploration tool, we are easily able to identify these differences. The first indicator that the file is a .NET Framework binary is that it will contain a field named the .NET Header within the PE File.

The .NET Header stores the relevant information about the binary relating to the .NET Framework version and specific conditions needed for the application to execute successfully on the host machine. The .NET Header also includes a bitwise-flag field named “Flags”, one of which indicates that this binary contains IL Only code. This flag identifies the binaries that we will be testing, and will be referenced throughout this post.

A second indicator that differentiates the x86, x64 and AnyCPU compiled binaries is the inclusion of a non-zero value AddressOfEntryPoint. The x64 binary AddressOfEntryPoint is set to zero as the Windows loader is able to identify that this is a .NET Framework binary from the .NET Header fields. AddressOfEntryPoint is only required for versions of Windows earlier than Windows XP (as we will discuss next), and Windows XP was the earliest version to support 64-bit processing. Therefore this field is unnecessary in the 64-bit binary.

The following table summarises whether key PE header fields are included for each target architecture, in all .NET Framework versions we tested:

Architecture PE Header Fields
AddressOfEntryPoint Import Table .NET Header
x64
x86
AnyCPU
Table 2: PE header fields that are present (i.e. non-zero) or absent in .NET Framework binaries built for different architectures. These results were consistent across binaries for all versions of the .NET Framework we checked, which encompassed v2.0 through to v4.8.

Loader operation

When Windows XP or newer detects that a .NET Framework executable is being loaded, the file MsCorEE.dll is loaded into memory. This provides the entry point to the runtime for .NET Framework EXE and DLL files (either _CorDllMain() or _CorExeMain() function, located in the MsCorEE DLL). However, this mechanism will not work under older versions of Windows, which rely on an executable’s entry point (a pointer in the header) to know how to launch the process. In order to support older versions of Windows, .NET Framework binaries built for either x86 or AnyCPU architecture also include a non-zero value for the AddressOfEntryPoint field in the Optional Header. This field is an RVA (Relative Virtual Address) to either the _CorDllMain() or _CorExeMain() function. The function that will be included in the Import Address Table is determined during compilation if the image file is either compiled as an executable or as a library. In either case, for x86 binaries the MsCorEE DLL essentially acts as a shim used to instantiate the CLR with the .NET Framework information gathered from the .NET Header.

The Microsoft documentation for both of those functions identifies a key function in the Windows loader chain named _CorValidateImage(). This function is called prior to _CorDllMain() or _CorExeMain(), while the image is being loaded into memory by the Windows loader and:

  • Checks if the image is managed (IL Only) or not
  • Updates the entry point to one of the _CorExeMain() / _CorDllMain() functions
  • Updates the image from x86 to x64 if required to do so

Now that we have established an understanding of how .NET Framework binaries are loaded, we can test how Microsoft Detours DLL injection runs under both Windows and Wine.

Methodology

Microsoft Detours allows us to inject a DLL into the Import Table of another application using the provided APIs DetourCreateProcessWithDllEx() and DetourCreateProcessWithDlls(). The Microsoft Detours repository provides a withdll.exe sample executable that allows us to perform this DLL injection using a variety of provided DLLs without a need to construct our own. This sample executable utilises the Detours API DetourCreateProcessWithDlls(), allowing us to inject an arbitrary number of DLLs into a target application while being loaded into memory.

As discussed earlier, an x64 .NET Framework binary does not include an Import Table like native applications do. This is not an issue here, as when Microsoft Detours loads an x64 .NET Framework image file into memory, an Import Table and Import Address Table will be created for us that will include the information of the injected DLL. As the x86 .NET Framework binaries include the Import Table already, the injected DLLs are added to the beginning of the list of Imported DLLs, ensuring they will be loaded first.

The sample application was used to identify any changes across different .NET Framework versions when observing the behaviour of injecting a DLL into a target application using Detours. We injected the simple.dll sample DLL for both the 32-bit and 64-bit tests, and the files were placed together in a directory with the filenames simple32.dll and simple64.dll. This bitness-based suffix naming scheme is required as Detours uses the suffix to identify the correct DLL (32-bit or 64-bit) to inject into a given target process.

This table gives an overview of the results:

.NET Framework Architecture Successfully Hooked by Detours
Test Application Detours Injected DLL Wine (Mono) Windows
v4.0 and older x64 x64
x86 x86
AnyCPU x64
v4.5 and newer x64 x64
x86 x86
AnyCPU x86
Table 3: Test results when attempting to hook .NET Framework executables with Detours under Wine and Windows.

This strongly suggests that the Wine loader is not behaving in the same way as the Windows loader, and something is preventing the injected DLL from being loaded. To help identify the loader behaviour on Wine we enabled trace logs for Module, PID, and TID in combination with Detours trace logging.

Examining the Wine logs allowed us to confirm that the injected DLL was being loaded into memory. When running a process through Detours the withdll.exe binary runs, and that launches the process we are modifying as a child process. Enabling the additional tracing to show us which processes were doing what allowed us to see that the parent process (withdll.exe) was mapping the injected DLL into memory to validate it, but the child process was not mapping the injected DLL at all. Examining the Detours logs confirmed that the Import Table was correctly being modified, and we can see the injected DLL being validated in the Wine logs. Since Detours is correctly modifying the Import Table, and since the DLL is correctly mapped into memory during the validation, it seems that the problem lies in how the Import Table is handled by the Wine loader.

To examine the differences between Windows and Wine loader behaviour in more detail, our colleague Adam Rehn created an automated test suite that runs a set of .NET Framework binaries in various configurations (both with and without Detours) and inspects the in-memory state of their PE headers at runtime. This allowed us to identify exactly which aspects of the Windows loader behaviour are missing in Wine’s implementation, and to devise a Wine patch to address these disparities.

Disparities and solutions

The Windows loader is tasked with loading an image file (EXE/DLL) into memory, and when a .NET Framework assembly is loaded it uses _CorValidateImage(), a function provided by MsCorEE.dll. _CorValidateImage() performs the validation of the image file contents and (if necessary) updates an x86 binary to x64 when loading it into memory before either _CorDllMain() or _CorExeMain() are called.

The Wine implementation of the loader does not call _CorValidateImage(), instead it directly performs a similar validation and verifies that the PE File meets the requirements for a .NET Framework binary by checking the fields in the .NET Header. Wine also converts an x86 file into an x64 equivalent with a call to convert_to_pe64(). Our investigation into the Wine loader logic revealed three areas in which it deviates from the behaviour of the Windows loader, although it turns out that only one of these actually led to the original problem we encountered of Detours not correctly hooking .NET Framework binaries. All three of these disparities have been resolved in the finished version of our Wine loader patch, which can be found here.

Import Table retention during 32-bit to 64-bit conversion

When an AnyCPU binary that does not specify the COMIMAGE_FLAGS_32BITPREFERRED flag is loaded into memory on a x64 machine, the convert_to_pe64() function converts the PE File Headers structure into the x64 equivalent. This conversion involves copying the header fields from the 32-bit structure into the 64-bit equivalent. For efficiency, under Wine only the necessary fields are copied. This currently excludes the Import Table fields, as they are not strictly needed for a .NET Framework application to run. This means that when an x86 binary is converted to x64 the Import Table will be blank, which differs from the behaviour of the Windows loader. Under Windows, the Import Table is visibly retained during the header conversion process.

We modified convert_to_pe64() to include IMAGE_DIRECTORY_ENTRY_IMPORT in the list of PE header directories which get copied over. This allowed us to match the Windows results when running the test applications without Detours, but it had no impact whatsoever on the Detours tests. After reviewing the Detours source code, we realised that Detours itself actually includes functionality to perform 32-bit to 64-bit conversion when required, and this logic runs before the OS loader has a chance to perform any conversion of its own. This means that convert_to_pe64() is not called when hooking binaries with Detours, and that the conversion logic differences between Windows and Wine are not the source of our original problem. So we removed the changes to convert_to_pe64() and continued investigating the other differences identified by the test results.

Imported DLL loading

When the Wine loader observes that the COMIMAGE_FLAGS_ILONLY flag is present in the .NET Header of a PE file, it will identify it as an IL Only binary and will call fixup_imports_ilonly() to process its imports. This function ignores the PE header Import Table entirely, and is hardcoded to instead just load the file MsCorEE.dll and resolve the function address for either _CorDllMain() or _CorExeMain(). This behaviour occurs for x86, x64 and AnyCPU .NET Framework binaries and prevents any imported DLLs from being loaded into memory.

Wine provides another fixup_imports() function which is responsible for loading in the images present in the Import Table of the given image. This function will recursively run through each subsequent image’s Import Table until all the required imports are loaded into memory if they are not already. However, this function is only ever called for native binaries, and when an IL Only binary is being loaded then fixup_imports_ilonly() is used instead.

To address the Wine loader not loading any Import Table entries for .NET Framework binaries, we included a call to fixup_imports() at the end of fixup_imports_ilonly() when the current image is confirmed to be an EXE by checking that the image file characteristics value does not include IMAGE_FILE_DLL. By doing this, all of the imports in the Import Table will be loaded into memory. To test whether loading the DLL files alone was sufficient, we included a check in import_dll() (which is called by fixup_imports()) to skip resolving the memory addresses of individual imported functions when processing the imports of an IL Only binary. With this change in place, we found that Detours was able to successfully hook .NET Framework executables!

Although we had confirmed that simply loading all of the imported DLLs was enough to fix the original problem, the test results still showed one more unexpected disparity between Windows and Wine loader behaviour, which was interesting enough to merit further investigation.

Imported function resolution

.NET Framework binaries will never actually refer to the memory addresses of individual functions listed in the Import Address Table, so it should not be necessary to resolve these addresses. Our testing with the logic to skip function resolution confirmed that Detours hooking works fine without any of the addresses filled in the Import Address Table, and the test results when running the test applications without Detours showed that Windows does not resolve these addresses either. But curiously, the results when running the test applications with Detours showed that the Windows loader actually was performing function resolution and filling the thunks in the Import Address Table.

The difference in Windows loader behaviour between test runs with and without Detours indicated that Detours itself must be making some sort of change to the PE headers to trigger a different code path. Reviewing the Detours source code once again, we noticed that Detours actually clears the COMIMAGE_FLAGS_ILONLY flag in the .NET Header. This explains why the Windows loader performs full import resolution like it would for a native binary, since presumably it checks for the presence of that flag. This raised a greater question though: if the COMIMAGE_FLAGS_ILONLY flag was no longer set, then why was Wine still calling fixup_imports_ilonly() rather than fixup_imports()?

As it turns out, Wine parses the flags in the .NET Header when the PE file is first mapped into memory, and then caches those parsed values and reuses them for subsequent checks. This means that any changes to the flags in the .NET Header are currently ignored. To address this, we modified our check at the end of fixup_imports_ilonly() to not just verify that the image is an EXE, but to also check the current in-memory state of the .NET Header and only call fixup_imports() if the COMIMAGE_FLAGS_ILONLY flag is no longer set. We then removed the check in import_dll() that skips function resolution, and the test results confirmed that Wine now matched the Windows behaviour exactly: the thunks in the Import Address Table are not filled for .NET Framework binaries when running without Detours, and they are filled with running with Detours.

With the correct fix in place for import resolution, we finally reintroduced our earlier changes in convert_to_pe64() to retain the Import Table when performing 32-bit to 64-bit conversion without Detours. With these combined changes, the test results for our finished patch now show that the Wine loader behaviour perfectly matches the Windows behaviour across all tests, both with and without Detours.

Final thoughts

Hooking system APIs gives us great insight into how they work, and enables us to perform various actions depending on our specific use case. Microsoft Detours helps us achieve this by providing a library that is easy to use, and provides a consistent and concise way of achieving this across the multiple versions of Windows and Wine. Detours works very well under both Windows and Wine for native binaries, and .NET applications, but unfortunately we were not able to hook .NET Framework executables under Wine.

We investigated how the .NET Framework differs from modern .NET, including the PE structure and how the underlying operating system loads these files into memory for execution. During this investigation we were able to identify key pieces of information in the .NET Header such as the flags used to identify if the application includes only managed code (IL Only). We were also able to identify the differences in the PE files created for the x86, x64 and AnyCPU target platforms and determine how those changes affect how they are loaded under Windows and Wine.

With this understanding of the structure of .NET Framework PE files, we were able to devise an empirical test suite which identified three key differences in loader behaviour between Windows and Wine. Our patch, available here, addresses all three of these discrepancies and ensures the Wine behaviour matches Windows perfectly across all of our tests. We are now successfully able to use Microsoft Detours to hook .NET Framework applications while running under Wine.

Ready to start something amazing?

Contact us today to discuss how we can provide the technology and expertise to make your vision a reality.

Contact Us