LFD445 Linux Kernel Debugging

Seminarinformationen

Seminar - Ziel

In diesem 3-tägigen Seminar „LFD445 Linux Kernel Debugging“ erwerben Sie praktische Kenntnisse zur Fehlersuche im Linux-Kernel. Sie lernen, wie Sie ein physisches oder virtuelles System für das Remote-Debugging von einem zweiten System aus einrichten. Zudem arbeiten Sie mit den grundlegenden Kernel-Komponenten, die den internen Debugging-Frameworks zugrunde liegen, und machen sich mit verschiedenen Techniken zur lokalen Analyse und Fehlerbehebung vertraut. Das Seminar bietet Ihnen das nötige Wissen, um Kernel-Probleme effizient zu identifizieren und zu debuggen.

Teilnehmer - Zielgruppe

Dieser Kurs richtet sich an aktuelle oder angehende Kernel-Entwickler, Treiber-Entwickler und alle, die den Entwicklungszyklus verkürzen möchten, indem sie bestehende Tools und Funktionen nutzen.

Kurs - Voraussetzungen

• Sehr gute Kenntnisse in Linux System Administration
• Erfahrung in C Programming
• Grundverständnis des Linux Kernel und Systemarchitektur
• Erfahrung mit Command Line Tools unter Linux
• Idealerweise Kenntnisse in System Programming
• Erfahrung auf dem Niveau des Kurses "LFD420 Linux Kernel Internals and Development""

Seminardauer

• 3 Tage
• 09:00 Uhr bis 17:00 Uhr

Schulungsunterlagen

• nach Absprache

Seminar-Inhalt / Agenda

Introduction

• Objectives
• Who You Are
• The Linux Foundation
• Copyright and No Confidential Information
• The Linux Foundation Training
• Certification Programs and Digital Badging
• The Linux Foundation THRIVE-ONE Subscriptions
• Linux Distributions
• Platforms
• Preparing Your System
• Using and Downloading a Virtual Machine
• Things Change in Linux and Open Source Projects
• Documentation and Links

Preliminaries

• Procedures
• Kernel Versions
• Kernel Sources and Use of git
• Hardware
• Staging Tree
• Labs

How to Work in OSS Projects **

• Overview on How to Contribute Properly
• Know Where the Code is Coming From: DCO and CLA
• Stay Close to Mainline for Security and Quality
• Study and Understand the Project DNA
• Figure Out What Itch You Want to Scratch
• Identify Maintainers and Their Work Flows and Methods
• Get Early Input and Work in the Open
• Contribute Incremental Bits, Not Large Code Dumps
• Leave Your Ego at the Door: Don't Be Thin-Skinned
• Be Patient, Develop Long Term Relationships, Be Helpful

Debugging Overview

• What is Debugging?
• OOPS and Panic Messages
• Output in dmesg
• Hidden Errors
• Course Flow

OOPS Messages

• What is an Oops Message?
• Finding the Code That Triggered the Oops
• Putting it All Together
• Background - Modules **
• Background - Module Compilation **
• Background - Macros and Inlined Functions **
• Lab Exercises

Oops Messages With Source Code

• Without Source Code
• With Source Code - Compiling
• With Source Code - Disassembly
• Summary

Kernel Features

• Components of the Kernel
• User-Space vs. Kernel-Space
• What are System Calls?
• Available System Calls
• Scheduling Algorithms and Task Structures
• Process Context
• Labs

Kernel Programming Preview

• Task Structure
• Memory Allocation
• Transferring Data between User and Kernel Spaces
• Object-Oriented Inheritance - Sort Of
• Linked Lists
• Jiffies
• Labs

Kernel Deprecated Interfaces

• Why Deprecated
• __deprecated
• BUG() and BUG_ON()
• Computed Sizes for kmalloc()
• simple_strtol() Family of Routines
• strcpy(), strncpy(), strlcpy()
• printk() %p Format Specifier
• Variable Length Arrays
• Switch Case Fall-Through
• Zero-Length and One-Element Arrays in Structs

Race Conditions and Synchronization Methods

• Concurrency and Synchronization Methods
• Atomic Operations
• Bit Operations
• Spinlocks
• Seqlocks
• Disabling Preemption
• Mutexes
• Semaphores
• Completion Functions
• Read-Copy-Update (RCU)
• Reference Counts
• Labs

Process Address Space

• Allocating User Memory and Address Spaces
• Locking Pages
• Memory Descriptors and Regions
• Access Rights
• Allocating and Freeing Memory Regions
• Page Faults
• Labs

Printk

• Debugging with printk
• Format Specifiers in printk
• no_hash_pointers Command Line Option
• Using early_printk
• Labs

Kernel Initialization

• Overview of System Initialization
• System Boot
• Das U-Boot for Embedded Systems**
• Kernel Startup

Monitoring and Debugging

• Debuginfo Packages
• Tracing and Profiling
• sysctl
• SysRq Key
• oops Messages
• Kernel Debuggers
• debugfs
• Labs

Ftrace

• What is ftrace?
• ftrace, trace-cmd and kernelshark
• Available Tracers
• Using ftrace
• Files in the Tracing Directory
• Tracing Options
• Printing with trace_printk()
• Trace Markers
• Dumping the Buffer
• trace-cmd
• Labs

Kernel and git Bisection

• What is kernel bisection?
• Bisection basics
• Scripting kernel bisection
• Labs

Kernel Development Tools

• KASAN
• KCSAN
• KFENCE
• Kmemleak
• KMSAN
• UBSAN
• Summary

Perf

• What is perf?
• perf list
• perf stat
• perf record
• perf report
• perf annotate
• perf top
• Labs

kprobes

• kprobes
• kretprobes
• SystemTap **
• Labs

eBPF

• BPF
• eBPF
• Installation
• bcc Tools
• bpftrace
• Labs

QEMU

• What is QEMU?
• Emulated Architectures
• Image Formats
• Third Party Hypervisor Integration
• Labs

Linux Kernel Debugging Tools

• Linux Kernel (built-in) tools and helpers
• kdb
• qemu+gdb
• kgdb: hardware+serial+gdb
• Labs

Crash

• Crash
• Main Commands
• Labs

Kernel gdb Scripts

• Kernel gdb Scripts
• Setup
• Commands
• Functions
• Labs

kexec

• kexec
• Kernel Configuration
• kexec-tools
• Using kexec
• Labs

Kernel Core Dumps

• Producing and Analyzing Kernel Core Dumps
• Labs

Closing and Evaluation Survey

• Evaluation Survey

Kernel Architecture I

• UNIX and Linux **
• Monolithic and Micro Kernels
• Object-Oriented Methods
• Main Kernel Components
• User-Space and Kernel-Space

System Calls

• What are System Calls?
• Available System Calls
• How System Calls are Implemented
• Adding a New System Call
• Labs

Kernel Style and General Considerations

• Coding Style
• Using Generic Kernel Routines and Methods
• Making a Kernel Patch
• sparse
• Using likely() and unlikely()
• Writing Portable Code, CPU, 32/64-bit, Endianness
• Writing for SMP
• Writing for High Memory Systems
• Power Management
• Keeping Security in Mind
• Labs

The proc Filesystem **

• What is the proc Filesystem?
• Creating and Removing Entries
• Reading and Writing Entries
• The seq_file Interface **
• Labs

Unified Device Model and sysfs

• Unified Device Model
• Basic Structures
• Real Devices
• sysfs
• kset and kobject examples
• Labs

Weitere Schulungen zu Thema Linux Foundation