Duration: 35 hours (5 days @ 7 hours per day)
This training program intends to equip Linux developers with necessary skills to develop, debug and diagnose issues within the Linux kernel-space.
Target Audience (Who should attend):
- Engineers working on Linux-based systems willing to get a
deeper understanding of the Linux kernel.
- Engineers intending to migrate onto Linux kernel for device
- Architects intending to understand how the Linux kernel works
to better leverage its capabilities into their projects.
- Fair knowledge of C programming and data-structures using C.
- Fair knowledge on UNIX/Linux systems and application programming.
- Fair knowledge of computer hardware architecture.
- Learn about Linux kernel architecture.
- Learn and understand the design intricacies of the Linux kernel.
- Learn about Linux kernel coding guidelines, best practices
and approaches towards development on the Linux kernel.
- Learn how to create custom patches and apply them on Linux
- Learn about various Linux memory management,
process management, filesystems, concurrency and
synchronization mechanisms within the kernel and best
practices for choosing them.
- Learn about Linux kernel interfaces
(procfs, sysfs, debugfs, syscalls)
- Learn about Linux kernel modules (LKM) and learn how to
create LKMs (for 3.x/4.x kernels)
- Learn how to customize Linux kernel by source
- Linux kernel debugging techniques covering kernel
debugging configuration parameters (lock debugging,
kmemleak detector), kprobes and remote debugging
- Learn various Linux kernel data-structures
and kernel-mode functions relevant to kernel
About the Instructor:
This training program will be handled by Mr. Chandrashekar Babu - an experienced FOSS technologist and corporate trainer with 20 years of experience in the IT industry. He has been a Linux enthusiast since late 1995 and has been a polyglot with diverse technology and programming language experience on Free Software/Open Source Software ecosystem.
He has been delivering training programs to some of the most prominent product companies on various technology domains (that also include Linux Kernel Internals / Device Drivers) since 2003.
You can know more about him by visiting the following URLs:
- Personal Website: http://www.chandrashekar.info/about
- LinkedIn Profile: http://in.linkedin.com/in/chandrashekarbabu
Training Cost: ₹ 15,000/- (inclusive of taxes)
How to register/enroll:
To enroll/register for this training program, kindly drop an
e-mail to firstname.lastname@example.org and await reply.
- Participants are expected to carry-in their laptops
with the following software installed and configured:
- VirtualBox 4.3.20 or above. The same can be
downloaded for free from http://www.virtualbox.org/
- VirtualBox 4.3.20 or above. The same can be
- A laptop with high-end configuration (Intel Core i5 or
above with atleast 4 GB of RAM) is recommended.
- Participants must ensure that atleast 16 GB of
free disk-space is available on their laptops - as this
is required for installing a pre-configured Linux setup
as a VirtualBox appliance for hands-on practice session
in this training program.
What will be provided at the end of training program ?
- At the end of the training program, participants can take
personal copies on their laptops/external hard drives
- Complete recording of screencasts along with audio of the
entire training sessions for future reference.
- Presentations, notes and code samples for topics being covered.
- The virtual machine appliance used during training sessions
for practising hands-on.
- Complete recording of screencasts along with audio of the
- A hard copy book on Linux Kernel Development by Robert Love
for each attending participant.
- Course completion certificate.
Slashprog Technologies, #C9 / C49, First floor, Mogappair Industrial Estate, Mogappair West, Chennai - 600 037.
How to reach venue:
- The closest prominent landmark to the venue is Vavin bus-stop towards
Ambattur Industrial Estate and HCL Technologies.
- Take the road that leads towards Mogappair West depot from Vavin signal.
- At approximately 350 metres, you would find Bata showroom, Grill Ministri and Fashion Look showroom.
- Take the road that leads to the right opposite to Fashion Look.
- Take the second right turn approximately 160 metres on this roads (opposite to Berton Tyres and Fosroc Chemicals).
- Enter the third building at the left on this street and walk upstairs to reach the Venue.
You can alternatively search for the following address on Google Maps:
C-9 Jeswant Nagar, Ambattur Industrial Estate Chennai, Tamil Nadu 600037 and use directions to reach there.
- April 13th 2015 (Monday) to April 17th 2015 (Friday).
- 9:30 AM to 5:30 PM [Lunch break between 1:00 PM to 2:00 PM]
NOTE: Lunch cannot not be arranged at the venue due to logistic reasons at this time. Please kindly co-operate.
NOTE: The below agenda details proposed topics for this training program. The flow of topics and the coverage will depend on wavelength of majority of attending participants for this training program. The focus will be more on ensuring that the attending participants gain better clarity of the fundamentals before proceeding towards advanced topics.
Introduction to Linux kernel
- Introductory overview on Linux ecosystem
- Design goals of the Linux kernel
- Linux kernel architectural overview
- Kernel-mode vs User-mode
- Features and benefits of Linux kernel design
- Monolithic kernel architecture
- Processes and Files
- "Everything is a file" paradigm
- Virtual memory using flat memory model
- Support for dynamically loadable kernel modules
- Typical flow of a Linux system call
- Linux kernel and process view of the system
- Identify some important documentation bundled with the Linux
- Understand the directory structure/organization of Linux
kernel source tree.
- From boot to init: a high-level code walkthrough of the Linux
kernel source to understand initialization sequence of various
subsystems during Linux kernel boot-up.
Memory management within the Linux kernel
- Memory management overview on Linux
- Virtual memory, demand paging and swapping
- Linux kernel data-structures to abstract memory
- Lazy-TLB support on Linux
- Zone allocator and related API
- Memory management mechanisms - SLOB vs. SLAB vs. SLUB allocator
- Using kmem_cache series of functions
- Using kmalloc functions
- Understanding the GFP flags
- Using vmalloc functions
- Creating custom emergency memory pools
- Choosing the right memory management functions and best practices
- Managing user-space memory access (
copy_to_user(), copy_from_user()and related functions).
Linux kernel modules [Demonstration / Hands-On]
- Linux kernel module organization
- Loadable device drivers and feature enhancement modules
- Managing Linux kernel modules
Linux kernel module development
- Important header files.
- Kernel module functions.
- Compiling LKMs.
- Exporting symbols from LKM.
- User libraries Vs. Kernel mode library functions.
- Module-specific macros and rudimentary functions.
- Handling module parameters.
printk()series of functions,
/proc/kmsgto read kernel log messages.
- Managing kernel logs using
- Tips, techniques and conventions used in writing programs within
the Linux kernel.
- Using kernel modutils tools (
- Creating simple kernel modules (for 3.x kernels).
- Using kernel mode memory management functions.
- Simple kernel module examples emphasizing on basic kernel-space
- A simple kernel module emphasizing on interfacing /proc
Linux process model
- Introduction to Linux processes.
- Process states and process groups.
- Processes Vs. Threads.
- Linux kernel threads.
- Linux kernel task structure.
- An overview on CFS scheduler.
- Process scheduling policies and mechanisms.
- Process management primitives -
- Scheduler related functions for managing processes and threads
from within kernel-space.
- Introduction to Linux filesystems and filesystem structure
- Linux Kernel VFS layers and architecture
- Linux file types, dentry and stat information
- Linux device model and device files.
- Pseudo filesystems (procfs, sysfs, debugfs)
- Write Linux kernel module to interface with /proc filesystem
- Write Linux kernel module to interface with debugfs
- Gather statistics/diagnostic information and manage processes
using files exposed by the kernel via /proc interface.
- Creating device files and understanding how they are wired to
Linux kernel execution contexts
- Process context
- Interrupt context
- SoftIRQ context
- Kernel functions -
Concurrency mechanisms within the Linux kernel
- Interrupt handlers
- Threaded IRQ handlers
- An overview on SoftIRQs
- Timer functions and hrtimer functions
- Using ktime, delay and sleep functions
- Tasklets and High-priority Tasklets
- Concurrency Managed Workqueues (and delayed workqueues)
- Best practices for designing module to differ work within
Synchronization mechanisms within the Linux kernel
- Memory barriers
- Per-CPU variables and Atomic variables
- Spinlocks, reader-writer locks
- Seqlocks and RCU
- Semaphores and mutexes
- Completions and wait-queues
- Notifier chains
- Best practices for choosing the right synchronization
mechanism for a given requirement.
- Simple kernel modules focussing on tasklets, timers,
work-queues and kthreads.
- Simple kernel modules focussing on synchronization
mechanisms using wait-queues, semaphores and completions.
- Using notifier chains within the Linux kernel
- Gathering statistics and managing IRQ lines using /proc/irq/
Debugging techniques on the Linux kernel
- Constraints and limitations involved in debugging the
- Understanding Kernel Support for Debugging
Debugging using printing
printk()series of functions.
pr_debug()and dynamic printk functions.
- Redirecting console messages.
- How messages get logged
- Turning messages on and off
- Rate limiting of console messages
- Debugging faults within Linux kernel.
- Deciphering Oops messages.
Using the Debug Filesystem
- Mounting and analyzing debugfs.
- Techniques and best practices for exposing kernel
variables and data-structures using debugfs for inspection
Debugging and instrumentation techniques using kprobe
- Introduction to kprobes
- Using kprobes, jprobes and kretprobes
- Performance analysis and intrumentation techniques
- An overview on Linux tracers
- Using function tracers to trace and instrument various
- Using MMIO tracer to trace all MMIO operations on memory-mapped
- An overview on blktrace, LatencyTOP, SystemTap and other tools
built on-top of Linux kernel tracers.
An overview on KGDB
- Configuring remote debugging using KGDB via serial
- Using KGDB with KDB integrated support.
- Setting breakpoints and analyzing kernel variables
using GDB console.
- Known issues and limitations with KGDB.
- Using KDB interface and writing custom KDB plugins.
Other debugging features within the Linux kernel
- Using the Magic SysRq key
- Using hardware breakpoint interface to detect potential
- Using kmemleak to check potential memory leaks.
- Using kexec and kdump to analyze kernel crash dumps.
Building Linux kernel from the sources (Optional *)
* Based on time availability
- Steps involved in deploying custom compiled Linux kernel from
the available sources.
- A quick walkthrough on Linux kernel source configuration parameters
- Linux kernel optimization techniques
- Speed/Performance optimization
- Size optimization
- Security optimization (Kernel hardening)
- Feature optimization
- Debugging options in the Linux kernel
- Creating custom boot image with busybox user-land.
- Using printk() for debugging
- Stack backtrace analysis from kernel crash
- Using Magic SysRq
- Writing simple module to expose variables and
structures to user-mode using debugfs
- Writing kprobe modules to analyze kernel
function arguments and return values
- Remote debugging using kgdb