Reverse Engineering Basics

A company finds a small tool on its own server and nobody remembers who wrote it. There is no source code, only the program. Before anyone runs it, somebody has to read what it does. That is reverse engineering: working out how a program works from the program itself.

• Binary: a program as the computer runs it, a file of machine instructions.
• Disassembler: a tool that turns machine instructions back into assembly you can read.
• Debugger: a tool that runs a program, stops it where you say and shows its memory.
• Register: a tiny store inside the processor. On this machine rdi holds the first argument of a call.
• Library call: a call to ready-made code such as puts, which prints text.
• System call: a request from a program to the operating system, such as writing.

The lab keeps its programs in the revlab folder. hello holds no secret: it prints one line. The file command says what kind of file it is without running it. Notice ELF, the format of programs on Linux, the processor it was built for, and not stripped, which means the names of its functions are still inside.

A program carries its text inside it, plain and readable, whether or not anybody meant to show it. strings prints every run of readable characters, which is a long list, so the command keeps the line with Hello in it. You find out what the program says without running it.

objdump -d turns the instructions into assembly, one per line. The command asks for the function main only and keeps the lines that mention puts: there is one, a call to the library function that prints. radare2 does the same job and adds names. aa asks it to analyse the file, pdf @ main prints the function. Look for the name puts and for the text it is given.

Run the program under gdb and stop it on the first line of puts. By the way a call works on this machine, puts finds the address of its text in the register rdi. x/s $rdi means: show the string at that address. So you read the argument of a function while the program is paused, the way you will read a password.

ltrace prints each library call with its arguments, and strace prints each system call. The first shows puts being given the text. The second shows the program asking the operating system to write the same bytes to the screen, using write. The greeting is sent to nowhere here so that only the trace is shown. Neither one needs to read any assembly.