DebuggingProgramCrash

Differences between revisions 1 and 111 (spanning 110 versions)
Revision 1 as of 2005-05-28 20:36:44
Size: 1040
Editor: adsl-213-190-44-43
Comment: imported from the old wiki
Revision 111 as of 2011-06-20 17:03:54
Size: 6438
Editor: pool-108-20-137-228
Comment: Prune pre-hardy details
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= DebuggingProgramCrash = <<Include(Debugging/Header)>>
||<tablestyle="float:right; font-size: 0.9em; width:40%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;"><<TableOfContents>>||
Line 3: Line 4:
 1. Check if the package has a debugging version available. In general, debugging packages will be named with a `-dbg` suffix
 1. If not, you can generally* build one this way:
  a. Install the build-time dependencies for the package: {{{
sudo apt-get build-dep <package>
This document describes how to install debug packages on Ubuntu, which will aid in providing information for bugs.

== Debug Symbol Packages ==

First, check if there is a package with a -dbg suffix in the main Ubuntu repositories. These are the debug symbol packages, and are equivalent to '-dbgsym' described below. You can safely use either one, but not both at once. Try installing the package, adding -dbg to its name, for example:
{{{
sudo apt-get install xserver-xorg-core-dbg
Line 8: Line 12:
  a. Build .debs for debugging: {{{
DEB_BUILD_OPTIONS="nostrip noopt" sudo apt-get source -b <package>
if the package in question is the ''xserver-xorg-core'' package.

If there is no -dbg package:

 1. Create an `/etc/apt/sources.list.d/ddebs.list` by running the following line at a terminal:
 {{{
echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiverse" | \
sudo tee -a /etc/apt/sources.list.d/ddebs.list
Line 11: Line 21:
  a. Install the needed .debs (they will be in the current working directory if the build succeeded): {{{
sudo dpkg --install <package.deb>
 1. Stable releases (not alphas and betas) require three more lines adding to the same file, which is done by the following terminal command:
 {{{
echo "deb http://ddebs.ubuntu.com $(lsb_release -cs)-updates main restricted universe multiverse
deb http://ddebs.ubuntu.com $(lsb_release -cs)-security main restricted universe multiverse
deb http://ddebs.ubuntu.com $(lsb_release -cs)-proposed main restricted universe multiverse" | \
sudo tee -a /etc/apt/sources.list.d/ddebs.list
Line 14: Line 28:
 1. Start the program under control of `gdb`: {{{
gdb <program>
(gdb) run <arguments, if any>

 You may also add these lines using the Synaptic Package Manager:
  * Choose ''Synaptic'' via the ''System > Administration'' menu.
  * Choose ''Software Sources'' or ''Repositories'' via the ''Settings'' menu, and click on the ''Third-Party Software'' tab.
  * Click the ''Add'' button and enter each ''deb ...'' line as above one by one and click the ''Add Source'' button (you will have to add these lines one at a time).

 1. Import the debug symbol archive signing key:
 {{{
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 428D7C01
Line 18: Line 38:
 1. The program will start. Perform any actions necessary to reproduce the crash
 1. Retrieve a backtrace of the crash: {{{
(gdb) thread apply all bt full }}}
Line 22: Line 39:
 1. Include the complete output from gdb in your bug report  1. Then run
 {{{
sudo apt-get update
}}}
 to update your package list or click the ''Reload'' button if you used the Synaptic Package Manager.
Line 24: Line 45:
* most of the packages support the build of debugging version in this way. If that doesn't work for a package you can open a bug against it.  1. The debug symbol packages have the '-dbgsym' suffix attached, so to install the debug symbols for the package 'yelp', you first run:
 {{{
apt-cache policy yelp
}}}
 This will show you the version number currently installed (we'll use 2.22.1-0ubuntu2.8.04.1 in this example).
 '''NOTE:''' 'yelp' [which is the package name of "Help and Support" located under the 'System' menu] is ''just an example'' and '''not''' a part of the command: 'yelp' is only used to demonstrate the procedure. You must replace 'yelp' with the name of the package you want to debug.

 1. Install the debug symbols:
 {{{
sudo apt-get install yelp-dbgsym=2.22.1-0ubuntu2.8.04.1
}}}
You can also use the Synaptic Package Manager to search for 'yelp-dbgsym' and install it from there.

The above procedure will install the debug symbol package for yelp only. Chances are that yelp uses shared libraries in other packages and debug symbols for them might be required in order to obtain a readable stack trace. You can download the [[attachment:list-dbgsym-packages-v2.sh]] shell script and run it (after a chmod a+x list-symbols-packages-v2.sh) this way:

/!\ we updated [[attachment:list-dbgsym-packages-v2.sh]] to list-symbols-packages-v2.sh. This is the first change to make it compatible with newer GDB (which does not, by default, load all libraries at start anymore). -v2 is currently only working if you run it against a currently-executing binary (i.e., with '-p $(pidof <whatever>').

 {{{
bash list-symbols-packages-v2.sh yelp
}}}
in order to obtain a more complete list of symbol packages to install. The script may also attach to a running process and in this case it gives an even more complete list of symbol packages to install, and this is recommended if the problem you are having does not prevent you from running the program. You can usually get the process ID of a running process using <code>pidof</code>, so while you have yelp running you might run:
 {{{
bash list-symbols-packages-v2.sh -p $(pidof -s yelp)
}}}
to get a list of packages that should be installed for best debug information. If the running process is a server (daemon), you might have to run 'list-symbols-packages.sh' with sudo.

 a. Now you make a [[Backtrace]].
 a. You can also run [[Valgrind]], if the program crashes with a "Segmentation fault" or "Bus error".
 a. Optionally, you may be asked to produce an [[Strace]].
 a. You can also provide this file : '''~/.xsession-errors'''

=== How do we remove all this stuff after getting the trace and get back to a normal system? ===

You can remove all debug symbol packages and the ddebs repositories with the following commands:

 {{{
sudo apt-get remove \.*-dbgsym
sudo rm /etc/apt/sources.list.d/ddebs.list
sudo apt-get update
}}}


=== References ===

 * Announcement: https://lists.ubuntu.com/archives/ubuntu-devel-announce/2006-September/000195.html

== The Xorg server ==
The X server will by default trap its own crashes and dump a stack trace in /var/log/Xorg.0.log. However, this stack trace is modified by the signal handler itself. To get a "normal" crash, which will trigger a core dump (and ''apport'' reporting), add this to your /etc/X11/xorg.conf:
{{{
Section "ServerFlags"
        Option "NoTrapSignals" "true"
EndSection
}}}
Please see DebuggingXorg for how to debug Xorg server crashes.

== Info for the BugSquad ==

If you're trying to `apport-retrace` a crash report from a bug that didn't happen on the same Ubuntu release as the one you're running, do the following:

Say that you're running `Gutsy` and the crash happened on `Feisty`:

 0. This will create a minimal `feisty` system. {{{
sudo mkdir -p /chroots/feisty
sudo debootstrap feisty /chroots/feisty/}}}
 0. Now you change into this minimal `feisty` system. {{{
sudo chroot /chroots/feisty}}}
 0. edit `/etc/apt/sources/list` and all the repositories you need, especially Martin's ddeb repository.
 0. {{{
apt-get update; apt-get install gdb apport}}}
 0. use `apport-retrace` as you're used to.

For more information on dealing with bug reports [[Apport]] see [[Bugs/ApportRetraces]].

----
CategoryBugSquad CategoryDebugging

Debugging Central

This page is part of the debugging series — pages with debugging details for a variety of Ubuntu packages.

This document describes how to install debug packages on Ubuntu, which will aid in providing information for bugs.

Debug Symbol Packages

First, check if there is a package with a -dbg suffix in the main Ubuntu repositories. These are the debug symbol packages, and are equivalent to '-dbgsym' described below. You can safely use either one, but not both at once. Try installing the package, adding -dbg to its name, for example:

sudo apt-get install xserver-xorg-core-dbg

if the package in question is the xserver-xorg-core package.

If there is no -dbg package:

  1. Create an /etc/apt/sources.list.d/ddebs.list by running the following line at a terminal:

    echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiverse" | \
    sudo tee -a /etc/apt/sources.list.d/ddebs.list
  2. Stable releases (not alphas and betas) require three more lines adding to the same file, which is done by the following terminal command:
    echo "deb http://ddebs.ubuntu.com $(lsb_release -cs)-updates main restricted universe multiverse
    deb http://ddebs.ubuntu.com $(lsb_release -cs)-security main restricted universe multiverse
    deb http://ddebs.ubuntu.com $(lsb_release -cs)-proposed main restricted universe multiverse" | \
    sudo tee -a /etc/apt/sources.list.d/ddebs.list
    You may also add these lines using the Synaptic Package Manager:
    • Choose Synaptic via the System > Administration menu.

    • Choose Software Sources or Repositories via the Settings menu, and click on the Third-Party Software tab.

    • Click the Add button and enter each deb ... line as above one by one and click the Add Source button (you will have to add these lines one at a time).

  3. Import the debug symbol archive signing key:
    sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 428D7C01
  4. Then run
    sudo apt-get update

    to update your package list or click the Reload button if you used the Synaptic Package Manager.

  5. The debug symbol packages have the '-dbgsym' suffix attached, so to install the debug symbols for the package 'yelp', you first run:
    apt-cache policy yelp
    This will show you the version number currently installed (we'll use 2.22.1-0ubuntu2.8.04.1 in this example).

    NOTE: 'yelp' [which is the package name of "Help and Support" located under the 'System' menu] is just an example and not a part of the command: 'yelp' is only used to demonstrate the procedure. You must replace 'yelp' with the name of the package you want to debug.

  6. Install the debug symbols:
    sudo apt-get install yelp-dbgsym=2.22.1-0ubuntu2.8.04.1

You can also use the Synaptic Package Manager to search for 'yelp-dbgsym' and install it from there.

The above procedure will install the debug symbol package for yelp only. Chances are that yelp uses shared libraries in other packages and debug symbols for them might be required in order to obtain a readable stack trace. You can download the list-dbgsym-packages-v2.sh shell script and run it (after a chmod a+x list-symbols-packages-v2.sh) this way:

Warning /!\ we updated list-dbgsym-packages-v2.sh to list-symbols-packages-v2.sh. This is the first change to make it compatible with newer GDB (which does not, by default, load all libraries at start anymore). -v2 is currently only working if you run it against a currently-executing binary (i.e., with '-p $(pidof <whatever>').

  • bash list-symbols-packages-v2.sh yelp

in order to obtain a more complete list of symbol packages to install. The script may also attach to a running process and in this case it gives an even more complete list of symbol packages to install, and this is recommended if the problem you are having does not prevent you from running the program. You can usually get the process ID of a running process using <code>pidof</code>, so while you have yelp running you might run:

  • bash list-symbols-packages-v2.sh -p $(pidof -s yelp)

to get a list of packages that should be installed for best debug information. If the running process is a server (daemon), you might have to run 'list-symbols-packages.sh' with sudo.

  1. Now you make a Backtrace.

  2. You can also run Valgrind, if the program crashes with a "Segmentation fault" or "Bus error".

  3. Optionally, you may be asked to produce an Strace.

  4. You can also provide this file : ~/.xsession-errors

How do we remove all this stuff after getting the trace and get back to a normal system?

You can remove all debug symbol packages and the ddebs repositories with the following commands:

  • sudo apt-get remove \.*-dbgsym
    sudo rm /etc/apt/sources.list.d/ddebs.list
    sudo apt-get update

References

The Xorg server

The X server will by default trap its own crashes and dump a stack trace in /var/log/Xorg.0.log. However, this stack trace is modified by the signal handler itself. To get a "normal" crash, which will trigger a core dump (and apport reporting), add this to your /etc/X11/xorg.conf:

Section "ServerFlags"
        Option "NoTrapSignals" "true"
EndSection

Please see DebuggingXorg for how to debug Xorg server crashes.

Info for the BugSquad

If you're trying to apport-retrace a crash report from a bug that didn't happen on the same Ubuntu release as the one you're running, do the following:

Say that you're running Gutsy and the crash happened on Feisty:

  1. This will create a minimal feisty system.

    sudo mkdir -p /chroots/feisty
    sudo debootstrap feisty /chroots/feisty/
  2. Now you change into this minimal feisty system.

    sudo chroot /chroots/feisty
  3. edit /etc/apt/sources/list and all the repositories you need, especially Martin's ddeb repository.

  4. apt-get update; apt-get install gdb apport
  5. use apport-retrace as you're used to.

For more information on dealing with bug reports Apport see Bugs/ApportRetraces.


CategoryBugSquad CategoryDebugging

DebuggingProgramCrash (last edited 2022-12-20 22:13:06 by sergiodj)