Decompiler Overhaul#568
Open
lazorr410 wants to merge 3 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a major overhaul of the DXIL-to-HLSL decompiler by adding new decompilation strategies (structural, stackify, mermaid-based) and an optional semantic-labeling post-pass, with corresponding CLI flags to select these modes.
Changes:
- Added a new structural code generation path with deferred targets + branch folding for improved SM6 coverage.
- Added a “stackify” recursive nesting emitter and a Mermaid CFG output mode for analysis/visualization.
- Added an optional semantic labeling post-pass and expanded CLI option parsing/help output.
Reviewed changes
Copilot reviewed 5 out of 8 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/utils/shader_decompiler/structural.hpp | New structural codegen path (branch folding, deferred targets, loop handling). |
| src/utils/shader_decompiler/stackify.hpp | New stackify codegen path (recursive nesting + phi_sel convergence handling). |
| src/utils/shader_decompiler/semantic_labels.hpp | New semantic labeling post-pass that annotates decompiled HLSL with inferred meanings. |
| src/utils/shader_decompiler/mermaid.hpp | New Mermaid flowchart/plan output for CFG visualization and debugging. |
| src/decompiler/cli.cpp | Adds CLI flags/help text and wires new decompilation options + semantic labeling. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+7
to
+13
| #include <map> | ||
| #include <regex> | ||
| #include <set> | ||
| #include <sstream> | ||
| #include <string> | ||
| #include <string_view> | ||
| #include <vector> |
Comment on lines
+157
to
+159
| if (!shortened.empty() && std::isupper(shortened[0])) { | ||
| shortened[0] = std::tolower(shortened[0]); | ||
| } |
| std::string field = m[3].str(); | ||
| std::string swizzle = m[4].matched ? m[4].str() : ""; | ||
| // Only label if it looks like a cbuffer access (capitalized name, known cbuffer names) | ||
| if (std::isupper(cbv[0]) && field.length() > 2) { |
| if (shared_predicates.empty() || continue_predicates.empty()) return std::nullopt; | ||
| std::stringstream shared_condition_stream; | ||
| for (size_t i = 0; i < shared_predicates.size(); ++i) { | ||
| if (i != 0) shared_condition_stream << " | "; |
| static void BuildDefUseChains(const std::vector<std::string>& lines, SemanticContext& ctx) { | ||
| // Match both: "float _42 = expr;" and "_42 = expr;" (mermaid mode reassignments) | ||
| static const std::regex ASSIGN_REGEX{R"(^\s*(?:\w+\s+)?(_\d+)\s*=\s*(.+);)"}; | ||
| static const std::regex VAR_REF_REGEX{R"((_\d+))"}; |
Comment on lines
+32
to
+37
| // Topological sort with loop body contiguity constraint. | ||
| // After emitting a loop header, all blocks dominated by the header | ||
| // (i.e., the loop body) must be emitted before any block outside the loop. | ||
| std::vector<int> topo_order; | ||
| std::set<int> visited; | ||
| std::set<int> in_queue; |
Comment on lines
+27
to
+37
| std::cerr << "Options:\n"; | ||
| std::cerr << " --flatten, -f Optimize/inline expressions\n"; | ||
| std::cerr << " --use-do-while Use do-while convergence wrappers\n"; | ||
| std::cerr << " --stackify Recursive if/else nesting with phi_sel\n"; | ||
| std::cerr << " --structural ShortFuse structural analysis (RECOMMENDED)\n"; | ||
| std::cerr << " --mermaid-decompile Condition-based flat guarded emission\n"; | ||
| std::cerr << " --mermaid Output Mermaid flowchart instead of HLSL\n"; | ||
| std::cerr << " --annotate Add block boundary and condition comments to output\n"; | ||
| std::cerr << " --annotate-mermaid Add full graph IR dump (nodes, edges, phi assignments)\n"; | ||
| std::cerr << " --semantic-labels Add semantic labels as inline comments\n"; | ||
| std::cerr << " --skip-existing, -s Skip if output file exists\n"; |
…fer access, fix a comparator, fix several logic issues, regex fixes
Comment on lines
+1633
to
+1642
| std::stringstream shared_condition_stream; | ||
| for (size_t i = 0; i < shared_predicates.size(); ++i) { | ||
| if (i != 0) shared_condition_stream << " | "; | ||
| shared_condition_stream << shared_predicates[i]; | ||
| } | ||
| std::stringstream continue_condition_stream; | ||
| for (size_t i = 0; i < continue_predicates.size(); ++i) { | ||
| if (i != 0) continue_condition_stream << " && "; | ||
| continue_condition_stream << continue_predicates[i]; | ||
| } |
Comment on lines
+7
to
+14
| #include <map> | ||
| #include <regex> | ||
| #include <set> | ||
| #include <sstream> | ||
| #include <string> | ||
| #include <string_view> | ||
| #include <vector> | ||
|
|
Comment on lines
+212
to
+213
| static const std::regex ASSIGN_REGEX{R"(^\s*(?:\w+\s+)?(_\d+)\s*=\s*(.+);)"}; | ||
| static const std::regex VAR_REF_REGEX{R"((_\d+))"}; |
Comment on lines
+257
to
+268
| for (const auto& user : ctx.def_use[var]) { | ||
| // Don't overwrite higher-confidence labels | ||
| if (ctx.labels.contains(user) && ctx.labels[user].confidence >= source_label.confidence * decay) { | ||
| continue; | ||
| } | ||
| // Only propagate through simple assignments and swizzles | ||
| // More complex expressions get pattern-based labels instead | ||
| if (ctx.use_def[user].size() == 1) { | ||
| ctx.labels[user] = {user, source_label.label + "_derived", source_label.confidence * decay, "propagated from " + var}; | ||
| next_worklist.push_back(user); | ||
| } | ||
| } |
Comment on lines
+28
to
+37
| std::cerr << " --flatten, -f Optimize/inline expressions\n"; | ||
| std::cerr << " --use-do-while Use do-while convergence wrappers\n"; | ||
| std::cerr << " --stackify Recursive if/else nesting with phi_sel\n"; | ||
| std::cerr << " --structural ShortFuse structural analysis (RECOMMENDED)\n"; | ||
| std::cerr << " --mermaid-decompile Condition-based flat guarded emission\n"; | ||
| std::cerr << " --mermaid Output Mermaid flowchart instead of HLSL\n"; | ||
| std::cerr << " --annotate Add block boundary and condition comments to output\n"; | ||
| std::cerr << " --annotate-mermaid Add full graph IR dump (nodes, edges, phi assignments)\n"; | ||
| std::cerr << " --semantic-labels Add semantic labels as inline comments\n"; | ||
| std::cerr << " --skip-existing, -s Skip if output file exists\n"; |
Comment on lines
+80
to
+82
| bool mermaid_decompile = std::ranges::any_of(arguments, [](const std::string& argument) { | ||
| return (argument == "--mermaid-decompile"); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds multiple new options for the decompiler:
--structural
--mermaid_decompile
--stackify
Structural is based on latest work from ShortFuse with modifications to handle many different edge cases with certain branching patterns and conditionals. It is the best path for producing HLSL with the most likeness to the original. The other two options still have failure cases on certain patterns.
The decompiler should now handle the vast majority of SM6 shaders, including large RayTracing shaders (~21000 line shader from Pragmata works fine for example).
I kept the other two options as mermaid kind of shows proof of concept and there was a lot of time put into both mermaid and stackify. They could potentially provide useful insights for future efforts.
The exe has been shared several times and its probably time to get the code out of just my local.