Binary Ninja Blog

Hello, and welcome to another exciting episode of “Sidekick in Action”! Today, we will be applying Sidekick to a variety of common vulnerability discovery tasks while looking at the popular network utility dnsmasq.

The Challenge

We have selected dnsmasq version 2.80 (stripped) as our target to analyze using Sidekick for several reasons:

• Historical Vulnerabilities: dnsmasq 2.80 has known CVEs, providing a clear, documented set of vulnerabilities for analysis.
• Realistic Deployment Scenario: The stripped binary, devoid of function symbols, mirrors real-world deployed software. The absence of symbols forces analysts to rely on code patterns, heuristics, and context, which illustrates how Sidekick can accelerate vulnerability discovery even in less-than-ideal conditions.
• Compact yet Complex Codebase: Despite its relatively small size, dnsmasq integrates diverse functionalities (DNS, DHCP, TFTP), presenting a rich collection of code paths and program logic.
• Open Source: Source code is available to enhance analysis if needed and to verify results.

Reverse engineering and vulnerability research involve non-trivial tasks like function identification, decompilation, pattern recognition, and vulnerability identification that all require considerable time, effort, and expertise. Sidekick can help with these by automatically reasoning about code and detecting well-known patterns and structures to identify behaviors, generate names, and recognize complex properties of software such as security.

Stage 1: What is this thing?

When looking at a binary for the first time, it helps to begin with an understanding of what you’re dealing with at a high level in order to ground the context of further investigations. Sometimes you may already have prior knowledge about the binary, and other times not. As mentioned above, we are starting with a version of dnsmasq that has been stripped of function symbol names and also given a generic file name to prevent Sidekick from placing too much attention on metadata instead of the code itself.

Using the Analysis Console interface, let’s start with something simple and ask the Analysis Assistant what the binary is.

After Sidekick completes, you’ll notice a few things:

• The Analysis Assistant issued several queries for information in the binary in the form of “Search binary” tool requests based on its determination of the information it needed to complete your request.
• The “Search binary” tool requests use the Binary Ninja Query Language (BNQL) to retrieve objects from the binary. The Analysis Assistant automatically generated the appropriate query using the correct syntax.
• After analyzing the results of its search, Sidekick provides an answer to our request along with a summary of its supporting findings.

The output from each tool request can be viewed by expanding the message using the chevron button that appears when you hover over the message. This allows us to see the information the Analysis Assistant uses as context to complete our request.

Before moving on to the next step, it’s important to keep in mind that the Analysis Console is powered by large language models that inherently perform with some amount of indeterminism. This means that for a given request, you may not always get the same answer. Since the Analysis Assistant makes potentially multiple decisions on the overall strategy and specific steps taken to complete your task, there will be variation in its results. Knowing this, we have provided a mechanism for you to easily request the Analysis Assistant to retry any request by simply hovering over its message and clicking the “Retry” button that appears.

Let’s see how that works in practice.

After Sidekick completes this time, you’ll notice a few things:

• The Analysis Assistant used a different approach and employed different steps to complete our request. This time it used just the strings used in the main function to determine what the binary is.
• The Analysis Assistant came to the same conclusion.

With that said, let’s continue on our dnsmasq bug hunt. Now that we “officially” know what binary we’re looking at, let’s see if we can determine the specific version since that information will determine what (if any) known vulnerabilities exist.

Since the Analysis Assistant was so helpful with our first request, let’s try it out on this task.

Before continuing on to find the version number, let’s first take note of the highlighted link to address 0x4fc56. Yes, the Analysis Console supports highlighted links to functions, symbols, addresses, and even Analysis Indexes. Simply click on the link, and the main view will navigate the cursor to that location for a nice quality of life improvement.

Now back to the version number. Well, it appears that Sidekick found a string containing a version number. Let’s confirm that the string is actually used in the code and, if so, look at the code to potentially verify its use as a version number of the program. While we could pull up Binary Ninja’s Cross-References sidebar for the string to see where it’s used in the binary, let’s see what Sidekick can do with this task.

In this case, Sidekick used a BNQL query to find the one function that uses the version string and captured that function’s code (sub_1cfa0 passes the version string as an argument to the function sub_14d30). Sidekick also captured the code of another instance of the version number being used as additional context to aid in its analysis before coming to the final determination that the version string is in fact used as a version number during DNS bindings on initialization.

Now that we know what our binary is and its specific version, let’s see if we can find any known vulnerabilities.

Stage 2: Are there any known vulnerabilities?

Sidekick leverages the vast knowledge of large language models (LLMs) to help answer your questions about the binary, so for this step, we are going to see what vulnerabilities it knows of that are associated with this version of dnsmasq.

The first thing you will notice is that Sidekick immediately tried to search the binary for evidence of vulnerabilities despite our request for information from its existing knowledge. That’s because the Analysis Assistant is trained to focus on the binary and apply its knowledge and resources to solving your problems in the context of the given binary. Therefore, in cases where you want general knowledge of the world outside your specific binary, you may need to instruct Sidekick accordingly. It appears that Sidekick is aware of several vulnerabilities in dnsmasq 2.80. For the purposes of this exercise, we are going to focus on CVE-2020-25682. After a casual search on the Internet, we find these details about it:

CVE-2020-25682

A buffer overflow vulnerability was discovered in the way dnsmasq extracts names from DNS packets before validating them with DNSSEC data. An attacker on the network, who can create valid DNS replies, could use this flaw to cause an overflow with arbitrary data in a heap-allocated memory, possibly executing code on the machine. The flaw is in the `rfc1035.c:extract_name()` function, which writes data to the memory pointed by name assuming MAXDNAME*2 bytes are available in the buffer. However, in some code execution paths, it is possible `extract_name()` gets passed an offset from the base buffer, thus reducing, in practice, the number of available bytes that can be written in the buffer.

Reference: Red Hat Security

We can use this information to guide Sidekick’s search and analysis of the bug, so let’s go find it.

Stage 3: Finding the Relevant Function

The CVE tells us that the bug is in the rfc1035.c:extract_name() function. However, the binary that we’re working with does not have symbols, so it’s not as simple as searching in Binary Ninja’s symbol list for extract_name. In a situation like this, we have a few options:

• Use a version containing symbols
• Task Sidekick with finding the vulnerable function
• Match the binary against source code

Option 1: Use a Version Containing Symbols

This option is off the table since we decided upfront that we would use a binary stripped of symbols, so let’s give Option 2 a try.

Option 2: Task Sidekick with Finding the Vulnerable Function

We are going to let Sidekick tackle the manually-intensive task of searching the entire binary looking for extract_name. Binary Ninja lists 864 functions for this binary. Even with Sidekick’s automation, it would consume a moderate amount of time for an LLM to tackle one at a time. While we could absolutely kick off a script to do that, let’s speed things up by first filtering on functions that exhibit characteristics of the function we’re interested in finding. Without symbols and without the source code, we do not have much information to go off of. Therefore, let’s ask Sidekick for some information and help finding some filtering criteria based on what we know.

It appears that Sidekick is generally aware of extract_name and what it does. Let’s see if Sidekick has a rough idea of its function signature since knowing how many parameters and even types could help reduce the search space.

Well that’s helpful. Based on the types of operations and data formats that extract_name handles, it seems like there should be specific constants or character literals associated with the function, so let’s ask Sidekick about that.

Based on these results, we have a decent set of criteria with which to initially filter candidate functions:

• Functions with 5 or more parameters, and
• Functions that use constants 0x2e or 0xc0 (selecting just a couple that seem most relevant)

This is a task suited best for the Automation Workbench, which is intended to automate repetitive analysis tasks. After an initial prompt with exactly that criteria and some minor feedback to output the results to an Analysis Index named extract_name candidates, we have a script automatically generated by the Scripting Assistant that finds all functions that satisfy our filtering criteria and writes them to an Analysis Index.

With the updated Automation Workbench in Sidekick 3.0, you will notice the following:

• Scripts are generated as tools with metadata (at the top of the script) that describe various aspects of the script, including its name, description, and parameters. This allows the Analysis Console to create, register and execute them during its analysis.
• Scripts can accept input parameters that get assigned during execution. When running a script that defines input parameters from the Automation Workbench, a pop-up dialog will appear allowing the user to specify values for input parameters.

After running the script, we can see the filtered functions in the Analysis Index extract_name candidates.

Our next step is to analyze each candidate in more detail to determine which function is the real extract_name. For this task, we will write another script to go through each of the candidate functions separately and leverage an LLM to analyze them and determine how similar the function is to extract_name based on the function’s code and the LLM’s knowledge of the extract_name function.

To do this, we create a new, blank script from within the Automation Workbench and enter an initial description of what we want to accomplish within the Scripting Assistant interface. We provide specific instructions to use an LLMOperator for the task of determining similarity and also include some helpful information about extract_name that we gleaned during our prior conversation with the Analysis Assistant.

Initially, the Scripting Assistant generated a version of the script that added the function signature string and specific DNS constants as parameters to the LLMOperator object (similarity_operator). However, for the sake of demonstrating another feature of the Automation Workbench, we moved that information directly to the Operator specification. Within the Operator: similarity_to_dns_masq_extract_name tab, we inserted the additional contextual information directly into the “instructions” section for the LLM that performs this task. This shows the ability to customize LLM operator tasks and provide more context if needed.

Another feature to note is that the Scripting Assistant decided to create a parameter (index_name) to the script, which is the name of the Analysis Index containing the list of functions to analyze. When executing this script from the Automation Workbench, a pop-up dialog will open for you to enter the value of the parameter, which in our case is extract_name candidates. This allows the script to be re-run using a different set of inputs.

After this script completes, the extract_name match scores Analysis Index is populated with similarity scores for each candidate function.

As you can see, there are several functions that are tied for the top spot, which indicates that there probably is not enough information to definitely say which is the real extract_name. Therefore, let’s move to Option 3.

Option 3: Match the Binary Against Source Code

Like any good reverse engineer would do in this situation, we searched the Internet to find the source code for our version of dnsmasq. After downloading the source package from Ubuntu’s repository, we opened up rfc1035.c and examined the extract_name function. While several external tools exist that perform the task of matching binary to source code, we will be using Sidekick to do the job.

Since we already have a list of candidate functions in the Analysis Index extract_name candidates, we will create another Analysis Workbench script that reads from that Index and uses an LLMOperator to compute the similarity of a candidate function to the source code for extract_name. In order to make this process more generalizable to the task of comparing any set of functions against an input reference source code, we direct the Scripting Assistant to find functions that match the C implementation passed as a keyword argument to an LLMOperator and then provide additional feedback to iterate through functions from our extract_name candidates index.

You will notice that the Scripting Assistant decided to create script parameters for both the name of the index containing the functions to examine and also the reference C implementation. This allows the script to be customized at run-time for this type of binary-to-source matching task.

After running the script, specifying our inputs, and waiting for Sidekick to complete, will the real extract_name please stand up?

We have a clear winner - sub_ed80, which if you recall from above was one of the candidates tied for the top match using only heuristics.

Stage 4: Analyzing the Code for Vulnerabilities

Now that we have identified the right function for where the vulnerability should exist, let’s have the Analysis Assistant take a deeper look at the function to confirm its presence and analyze its behavior. First, let’s tell Sidekick the function signature from source (since we have that now) and ask it to clean up the decompilation.

Before:

After:

Based on the information in our request, Sidekick renamed the function and its parameters and created and applied a structure definition for the dns_header to the header parameter, demonstrating the ability of the Analysis Console to perform edits on the binary. Using the Binary Ninja Types sidebar, we can see the structure type definition for dns_header that the Analysis Console automatically created using the capabilities of an LLM.

Compare this to the structure definition for dns_header in the source file dns-protocol.h:

struct dns_header {
 u16 id;
 u8  hb3,hb4;
 u16 qdcount,ancount,nscount,arcount;
};

Nice work Sidekick!

Now, without any hints about the vulnerability, we ask Sidekick to find the vulnerability.

Without hints, Sidekick is able to correctly identify the source of CVE-2020-25682 based on its assessment of the Buffer Overflow Vulnerability, which is due to the code lacking a check to ensure the name buffer is large enough.

Now, let’s tell Sidekick about the specific vulnerability we’re interested in.

With hints, Sidekick confirms the existence of the vulnerability and identifies its location at 0xee50. It also is able to reason about how to fix the vulnerability.

Conclusion

In this exercise, we showcased the power and versatility of Sidekick 3.0 in tackling real-world binary analysis challenges. By analyzing the stripped dnsmasq 2.80 binary, we navigated through multiple stages - from initially identifying the binary and determining its version, to honing in on candidate functions using AI-informed filtering criteria, and finally, matching those functions against the known source code. The process not only demonstrated how Sidekick leverages advanced BNQL queries and automated scripts to reduce manual effort, but also how it integrates large language models to reason about and pinpoint vulnerabilities like CVE-2020-25682.

Ultimately, Sidekick’s ability to dynamically adapt its analysis by understanding what the user wants to accomplish, generating a plan to accomplish it, executing the steps for that plan, and reasoning about the results highlights its potential to streamline and enhance vulnerability research.

If you are not already using Sidekick, then sign up today to start making deep analysis of your binaries a breeze!