Skip to content

Implement Improved MaxStackCalculator#572

Open
PhilippNaused wants to merge 1 commit into
0xd4d:masterfrom
PhilippNaused:betterMaxStackCalculator
Open

Implement Improved MaxStackCalculator#572
PhilippNaused wants to merge 1 commit into
0xd4d:masterfrom
PhilippNaused:betterMaxStackCalculator

Conversation

@PhilippNaused

@PhilippNaused PhilippNaused commented Feb 13, 2026

Copy link
Copy Markdown
Contributor

I have refactored the MaxStackCalculator to be faster and not fail on obfuscated code.
The new implementation traverses the instructions the same way an IL interpreter would, using multiple recursion on branch instructions.
Here is the original: https://github.com/PhilippNaused/Vilens/blob/main/src%2FVilens%2FHelpers%2FStackHelper.cs

I tested this on multiple different libraries both before and after code obfuscation.
The old implementation would often fail on obfuscated code. This one doesn't.

The new version is also faster and uses less memory:

Method Mean Error StdDev Ratio RatioSD Gen0 Gen1 Allocated Alloc Ratio
old 5.243 μs 0.1259 μs 0.1884 μs 1.00 0.05 1.2894 0.0687 21.79 KB 1.00
new 3.460 μs 0.0594 μs 0.0889 μs 0.66 0.03 0.0725 - 1.23 KB 0.06

@PhilippNaused PhilippNaused force-pushed the betterMaxStackCalculator branch from dde0933 to c1386e0 Compare February 13, 2026 08:10
@PhilippNaused PhilippNaused marked this pull request as ready for review February 13, 2026 08:10
@PhilippNaused PhilippNaused force-pushed the betterMaxStackCalculator branch from c1386e0 to 8b1250d Compare February 15, 2026 13:50
@PhilippNaused

Copy link
Copy Markdown
Contributor Author

@wtfsck, @ElektroKill Any thoughts?

@ElektroKill

Copy link
Copy Markdown
Contributor

@wtfsck, @ElektroKill Any thoughts?

I have emailed wtfsck about this and the other pull request, as they are the ones with write access and can merge the PR.

@wtfsck

wtfsck commented May 31, 2026

Copy link
Copy Markdown
Contributor

The old impl just did what the spec said. Assuming no-one has a problem with the new impl, it could be merged.

@ElektroKill

Copy link
Copy Markdown
Contributor

The old impl just did what the spec said. Assuming no-one has a problem with the new impl, it could be merged.

The spec has been "augmented" by Microsoft to remove the statement "It shall be possible, with a single forward-pass through the CIL instruction stream for any method, to infer the exact state of the evaluation stack at every instruction." together with the entire III.1.7.5 Backward branch constraints section of the specification.

See: https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md#backward-branch-constraints

Therefore, the change to a more robust implementation has merit if the new implementation has been thoroughly tested and works well.

@PhilippNaused PhilippNaused force-pushed the betterMaxStackCalculator branch from 8b1250d to d563e79 Compare June 1, 2026 06:46
@PhilippNaused

Copy link
Copy Markdown
Contributor Author

@ElektroKill What would you suggest for testing?

@ElektroKill

Copy link
Copy Markdown
Contributor

@ElektroKill What would you suggest for testing?

A rather simple, but decently effective testing method would be to use the current dnlib max stack calculator on a few large binaries with various methods (think binaries like the core libraries, dnlib itself, dnSpy binaries, etc.), and then do the same with the new max stack calculator. Finally, compare whether the max stack was calculated properly and is the correct value (the older max stack calculator is assumed to be correct for the test).

@PhilippNaused

PhilippNaused commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

@ElektroKill OK, I wrote a small test script:

#:package Autofac@9.1.0
#:package AutoMapper@16.1.1
#:package Castle.Core@5.2.1
#:package Moq@4.20.72
#:package Newtonsoft.Json@13.0.4
#:package NUnit@4.6.1
#:package Serilog@4.3.1
#:project src\dnlib.csproj

using dnlib.DotNet;
using dnlib.DotNet.Writer;

var modules = new[] {
	typeof(Object).Module,
	typeof(Autofac.Module).Module,
	typeof(AutoMapper.Mapper).Module,
	typeof(Castle.Core.ProxyServices).Module,
	typeof(Moq.Mock).Module,
	typeof(Newtonsoft.Json.JsonConvert).Module,
	typeof(NUnit.Framework.Assert).Module,
	typeof(Serilog.Log).Module,
	typeof(System.Text.Json.JsonDocument).Module,
};

var methods = modules
	.Select(ModuleDefMD.Load)
	.SelectMany(m => new MemberFinder().FindAll(m).MethodDefs.Keys)
	.ToList();

foreach (var method in methods) {
	if (method.Body is null)
		continue;
	var stackHeight = MaxStackCalculator.GetMaxStack(method.Body.Instructions, method.Body.ExceptionHandlers);
	Console.WriteLine($"{method.FullName} -> {stackHeight}");
}

This will calculate the stack height for 64699 method bodies.
You can place it in the root of this repo and run it once on the master and once on my branch.
The result is the same.
e.g.:

git switch master
dotnet run Test.cs > before.txt
git switch betterMaxStackCalculator
dotnet run Test.cs > after.txt
git diff --no-index before.txt after.txt

@ElektroKill

ElektroKill commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Thank you for the test script. I think comparing the max stack on almost 65000 methods is a good enough indicator that the new implementation is alright.

Now it's time for @wtfsck's input and decision :D

Finally, @wtfsck, after this PR, could we have a new release made featuring all the new changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants