How to Debug Programs on Remote Server using GDBServer Example

Remote debugging is the process of debugging a program running on a different system (called target) from a different system (called host).

To start remote debugging, a debugger running on host machine connects to a program which is running on the target via network.

The debugger in the host can then control the execution of the program on the remote system and retrieve information about its state.

Remote debugging is often useful in case of embedded applications where the resources are limited.

In this tutorial, we will see how we can debug programs running on a different system using GDB Server.

If you are new to GDB, you should first understand how to use GDB to debug C program.

We need the following two utilities to perform a remote debugging.

• gdbserver – Run this on your target system
• GDB – Execute this on your host system to connect to your target system

GDB and gdbserver communicate via either a serial line or a network, using the standard gdb remote serial protocol.

1. Install gbdserver on Target System

Target machine is the one which is running the program which you have to debug. You need to have the “gdbserver” executable on the target machine.

$ sudo apt-get install gdbserver

To do remote debugging, start your program using the gdbserver. gdbserver then automatically suspends the execution of your program at its entry point, and it waits for a debugger to connect to it. gdbserver doesn’t need the symbols from your program to debug. So you can strip symbols out of your program binary to save space.

$ gdbserver localhost:2000 my_prg 

Process program created; pid = 2045
Listening on port 2000

The above command suspend the execution on my_prg, and waits for a debugger to connect to it on port 2000.

2. Launch gdb on Host System

The executable file and the libraries in the host, must exactly match the executable file and libraries on the target, with an exception that the target binary symbols can be stripped. You can also load the symbols separately in the host using “file” command in gdb.

Run GDB on the host.

$ gdb my_prg
(gdb)

Use “target remote” to connect to the target system.

(gdb) target remote 192.168.1.10:2000

Now you can run the normal gdb commands, as if you are debugging a local gdb program.

3. Remote Debugging Demo Example

The following C program example will be used to demonstrate the remote debugging.

#include <stdio.h>

int power(int,int);

int main() {

        int i;
        printf("Program to calculate power\n");
        for (i=0;i<10;i++)
                printf("%d %d\n",i, power(2,i));
        return 0;
}

int power (int base, int n) {

        int i,p;
        p=1;
        for (i=1; i<=n; i++)
                p = p*base;
        return p;
}

$ cc -g -o my_prg power.c

On Target Machine,

$ gdbserver localhost:2000 my_prg
Process my_prg created; pid = 20624
Listening on port 2000

On Host Machine,

$ gdb my_prg

(gdb) target remote 192.168.1.10:2000
Remote debugging using 192.168.1.10:2000
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00007ffff7dddaf0 in ?? () from /lib64/ld-linux-x86-64.so.2
(gdb) b main
Breakpoint 1 at 0x400550
(gdb) continue 
Continuing.

Breakpoint 1, 0x0000000000400550 in main ()

Now we have connected the gdb for remote debugging. In the last example, we have put a breakpoint in main() function. If we continue our program, the output of the program will be printed in the target machine.

On Host:

(gdb) continue

On Target:

Remote debugging from host 192.168.1.20
Program to calculate power
0 1
1 2
2 4
3 8
4 16
5 32
6 64
7 128
8 256
9 512

Child exited with status 0
GDBserver exiting

4. Attach gdb to a Running Process on Target

First you have to find the process ID of the running process in target.

On Host,

(gdb) attach 3850

Now the gdb will suspend the process 3850 in the target and you can debug the program using normal gdb commands.

5. Launch gdbserver in Multi-process Mode

In the previous demo, you would have noticed that once the program executed successfully, the gdbserver also got exited. In real-time, you may want to debug multiple programs remotely, and you may not want to start the gdbserver every time with different program names. Do the following to achieve that.

On Target, run the gdbserver with –multi and without a program name.

$ gdbserver --multi localhost:2000
Listening on port 2000

On Host,

$ gdb

(gdb) target extended-remote 192.168.1.10:2000
Remote debugging using 192.168.1.10:2000

(gdb) (gdb) set remote exec-file /my_prg
(gdb) file /my_prg 
Reading symbols from /my_prg...(no debugging symbols found)...done.
(gdb) b main
Note: breakpoint 1 also set at pc 0x400550.
Breakpoint 2 at 0x400550
(gdb) run
Starting program: /my_prg
Breakpoint 1, 0x0000000000400550 in main ()

From the above snippet,

1. ‘target extended-remote’ is used to run gdbserver in multi process mode.
2. ‘set remote exec-file /my_prg’ is used to set the program which you want to debug in the target.
3. ‘file /my_prg’ is used to load the debugging symbols from the program in the host.
4. ‘b main’ is used to set breakpoint at main() function.
5. ‘run’ is used to run the program, which stops at the breakpoint main().

Note: In the above case, the executable “my_prg” is present under “/” on both target and host.

Now either you can ‘continue’ or ‘detach’ the program from debugging. Still the gdbserver will not exit in the target machine, so you can change the ‘remote exec-file’ at any time, and debug a different set of program.