Apex Universal Library Kernel Module Installation
The purpose of this tutorial is to demonstrate the complete process of compiling and installing the Linux kernel module for use with the STX104 and Tracer boards. The kernel module is delivered as source which must be compiled to your target Linux Kernel which can be different depending on the Linux distribution (distro) you chose. Additional information can be found in the README.txt file found within the drv_aul_x86 project folder within the Apex Embedded Systems library/driver Eclipse workspace.
Note that if we say "driver" we mean the kernel module.
The kernel module resides in kernel space whereas the library resides on the application side. The library is a set of functions (APIs) that are typically statically compiled to the application; its purpose is to bury much of the details between the application and the kernel module. I like to think of the library as the "helpful engineer" to ensure that things run smoothly. The library does an extensive amount of verifying especially for configuration related tasks. If there are any floating point calculations, we attempt to perform those operations in the library as in the x86 world performing floating point calculations in kernel space lead to special issues with respect to floating point processors in older x86 environments where a new value from the floating point processor is emitted some time later after the request which leads to interrupt and semaphore handling which is best to avoid so as to not waste kernel time.
SETUP / INSTALLATION GUIDE
This section discusses how to get your system set up to properly compile the Linux driver.
Kernel-Headers
Installation of the Linux kernel headers. If kernel headers are not on your system that you intend to compile this kernel module, then you must download the kernel headers. Note: if you are logged in as root, then you do not need the "sudo" prefix on the command lines below. Note: If "sudo" does not work, try "su" and enter password. Thereafter, you are logged in as super-user. There maybe better ways of doing this to preserve security. This guide is more about just getting the driver installed and functional.
Make sure you have updated version of Linux:sudo apt-get updateInstall linux-header package for Debian or any Debian derivative (i.e. Ubuntu, etc.):
sudo apt-get install linux-headers-$(uname -r)If GCC is not installed on your system, you must install it now:
apt-get update && apt-get upgradeTo verify its installation:
apt-get install build-essential
whereis gcc makeTo determine version of gcc and make:
gcc -v
make -v
Compilation of The Kernel Module
The kernel module must be compiled to the specific Linux target you want to run the kernel module. It is easier if your development system Linux kernel matches your target Linux Kernel as it makes things a bit easier. However, it is possible to have a development system different than your target; it will take a bit more effort to setup but can be done.
Compilation of the driver using Eclipse IDE development.
Using Eclipse, simply right click on the the drv_aul_x86 and click on "Clean Project" followed by "Build Project". Note that this project is a Makefile project and will build the project (i.e. x86 kernel driver) using the Makefile. The Makefile can be found within the drv_aul_x86 folder.
From a terminal, we can also perform the same clean and build operations with similar reports found within the Eclipse console. Open a terminal and move to the drv_aul_x86 folder (change to the directory or location).
Remove old build copies:
mike@Ubuntu32vb:~/aes_ws/drv_aul_x86$ make clean <-- remove old build copies
Build the driver:
mike@Ubuntu32vb:~/aes_ws/drv_aul_x86$ make all <-- rebuild driver
echo "Drv_Test make - Compilation begin" Drv_Test make - Compilation begin
... <-- not all output shown!
/home/mike/aes_ws/drv_aul_x86/../aul_tree/auldrv/../io/io_private.h:56:10: note: #pragma message: aul_tree: io_private.h using kernel copy_to/from_user /home/mike/aes_ws/drv_aul_x86/../aul_tree/auldrv/auldrv.c:146:13: note: #pragma message: PDEBUG: __KERNEL__ and DRIVER_NAME=auldrv. LD [M] /home/mike/aes_eclipse/drv_aul_x86/mod_auldrv.o Building modules, stage 2. MODPOST 1 modules CC /home/mike/aes_ws/drv_aul_x86/mod_auldrv.mod.o LD [M] /home/mike/aes_ws/drv_aul_x86/mod_auldrv.ko make[1]: Leaving directory `/usr/src/linux-headers-3.2.0-75-generic-pae' echo "Drv_Test make - default build complete." Drv_Test make - default build complete.
Dump Makefile variable information dump:
mike@Ubuntu32vb:~/aes_ws/drv_aul_x86$ make echo <-- dump information
/**************************************** Apex Embedded Systems SVN_DATE = NAME = auldrv DRV_NAME = 'DRIVER_NAME=auldrv' DEBUG_ENABLE = -D_DEBUG KERNEL_SOURCE= /lib/modules/3.2.0-75-generic-pae/build PWD = /home/mike/aes_ws/drv_aul_x86 EXTRA_CFLAGS = -w -Wimplicit -Wstrict-prototypes -Wunused -O6 -march=i486 -D'DRIVER_NAME=auldrv' -D_DEBUG OBJS = ../aul_tree/str_safe/str_safe.o ../aul_tree/shared/debug.o ../aul_tree/shared/error.o ../aul_tree/shared/status.o ../aul_tree/shared/mem.o ../aul_tree/io/io.o ../aul_tree/io/io_linux_drv_gen_x86.o ../aul_tree/auldrv/auldrv.o HDR = ../aul_tree/str_safe/str_safe.h ../aul_tree/shared/debug.h ../aul_tree/shared/error.h ../aul_tree/shared/status.h ../aul_tree/shared/mem.h ../aul_tree/io/io.h ../aul_tree/io/io_shared.h
Note that the "make echo" is useful for purposes of troubleshooting the Makefile and understanding what is actually assigned to the make variables.
The Makefile
Below is a listing of the Makefile. Here is a brief description of a couple of the makefile variables that are used. This tutorial is broad brushing the Makefile, but we are illustrating several main points hereTo post printk() print functions within the kernel driver which are pumped into the dmesg buffer within the Linux kernel, then uncomment the DEBUG_ENABLE, line 12.
The name of the driver, which is pulled using the library function AUL_Driver_Name(), is set using the NAME parameter at line 25. Line 26 converts the NAME parameter into an actual string. The name of the driver cannot be changed, otherwise the library will get confused.
You can also see the sections "all" (line 66), "clean" (line 72) and "echo" (line 77) which were used above to clean, build and then dump the Makefile variables.
# # # AUL driver kernel module Makefile using I/O simulation (array of memory) # Apex Embedded Systems # SVN_DATE := '$Date: 2014-04-29 09:54:14 -0500 (Tue, 29 Apr 2014) $' # 012345678901234567890123456789012345678901234567890 # 0 1 2 3 4 5 ########################################################### # Uncomment to post printk() debug messages DEBUG_ENABLE := -D_DEBUG ########################################################### # Uncomment to simulate I/O activities as printk() messages #NOT IMPLEMENTED: SIMULATE_IO := -D IO_SIMULATION ########################################################### # The name of the driver and prefix name of the devices. # Driver module will be named mod_. Example: mod_auldrv.ko # # NOTE: There is no destinction between the driver running # the simulator versus # drivers read/writing directly to hardware. Later version will repair that! NAME := auldrv DRV_NAME := 'DRIVER_NAME="$(NAME)"' ########################################################### ########################################################### ########################################################### OBJS = \ ../aul_tree/str_safe/str_safe.o \ ../aul_tree/shared/debug.o \ ../aul_tree/shared/error.o \ ../aul_tree/shared/status.o \ ../aul_tree/shared/mem.o \ ../aul_tree/io/io.o \ ../aul_tree/io/io_linux_drv_gen_x86.o \ ../aul_tree/auldrv/auldrv.o HDR = ../aul_tree/str_safe/str_safe.h \ ../aul_tree/shared/debug.h \ ../aul_tree/shared/error.h \ ../aul_tree/shared/status.h \ ../aul_tree/shared/mem.h \ ../aul_tree/io/io.h \ ../aul_tree/io/io_shared.h EXTRA_CFLAGS := -w -Wimplicit -Wstrict-prototypes -Wunused -O6 #-fomit-frame-pointer -pipe EXTRA_CFLAGS += -march=i486 EXTRA_CFLAGS += -D$(DRV_NAME) $(DEBUG_ENABLE) $(SIMULATE_IO) # if KERNELRELEASE is defined, we've been invoked from the # kernel build system and can use its language. ifneq (${KERNELRELEASE},) obj-m := mod_$(NAME).o mod_$(NAME)-objs := $(OBJS) # Otherwise we were called directly from the command line. # Invoke the kernel build system. else KERNEL_SOURCE := /lib/modules/$(shell uname -r)/build #using the eclipse generated "Debug" folder with all make files. # AUL_SOURCE := ../aul_tree/_Out_Auldrv_Kernel_Module PWD := $(shell pwd) all: ${HDR} echo "Drv_Test make - Compilation begin" ${MAKE} -C ${KERNEL_SOURCE} M=${PWD} -I../aul_tree modules # ${MAKE} -C ${AUL_SOURCE} M=${PWD} all echo "Drv_Test make - default build complete." clean: rm ${OBJS} ${MAKE} -C ${KERNEL_SOURCE} SUBDIRS=${PWD} clean # ${MAKE} -C ${AUL_SOURCE} SUBDIRS=${PWD} clean echo: @echo @echo '/****************************************' @echo "Apex Embedded Systems" @echo "SVN_DATE = $(SVN_DATE:5)" @echo "NAME = $(NAME)" @echo "DRV_NAME = $(DRV_NAME)" @echo "DEBUG_ENABLE = $(DEBUG_ENABLE)" @echo "KERNEL_SOURCE= $(KERNEL_SOURCE)" @echo "PWD = $(PWD)" @echo @echo "EXTRA_CFLAGS = $(EXTRA_CFLAGS)" @echo @echo "OBJS = $(OBJS)" @echo @echo "HDR = $(HDR)" @echo @echo '****************************************/' endif
Loading the Kernel Module
To load the AUL kernel module (or driver) we can use the insmod Linux command. Refer to Helpful Linux Commands below for further details and definitions.
Here is an example to get you started. In this case we are assigning the STX104 to base address 0x300 while the Tracer is assigned to base address 0x310.
sudo insmod mod_auldrv.ko offset=0x300,0x310 bytes=0x10,0x10 acronym="stx104","tracere" type="stx104","tracere"
The insmod command, specific to the mod_auldrv kernel module, can accept the following parameters. These will be retained as defaults within the kernel driver so long as the kernel module remains installed in the kernel. These must be passed each time the kernel module is installed.
| PARAMETER | DESCRIPTION |
| major | This defines the major number of the driver. Typically, this will not be used with insmod and the driver will ask to have a major number dynamically assigned. Example: major=0x81 Note: we don't recommend assigning a major number unless you have too. |
| offset | This is an array of one or more hardware base addresses for the PC/104 board device(s). Examples: offset=0x300 <-- one board offset=0x300,0x310 <-- two boards Note: if DEVICE_QTY is set to one (1) and we try to insmod two boards, only one board will be initialized. |
| bytes | This is an array of one or more hardware I/O space required. In other words contiguous bytes within the I/O space. Examples: bytes=0x10 <-- one board using 16 contiguous I/O bytes. bytes=0x10,0x10 <-- two boards |
| acronym | This is an array of one or names that are used to 'open' then device(s). Examples: acronym="stx104" <-- one board acronym="stx104","tracere" <-- two boards |
| type | Defines the type of the device. Possible types are: "stx104", "tracere", "summit" Examples: acronym="stx104" <-- one board acronym="stx104","tracere" <-- two boards |
Complete Examples
Here are two very brief complete examples which include the installation of the kernel module (driver) and creation of a very simple application program to open and close the devices.
Example #1:
Single STX104 installed on a stack with base address at 0x300 stacked onto a PC104 stack.
Driver Installation:
sudo insmod mod_auldrv.ko offset=0x300 bytes=0x10 acronym="mystx104" type="stx104"
Using the library we open the device as follows:
int board;
error_code = AUL_Initialization( NULL );
if ( error_code ) return error_code;
error_code = AUL_Device_Open( "mystx104", &board );
if ( error_code )
{ printf( "Error code: %s\n", AUL_Last_Error_Get() );
AUL_Termination();
return error_code;
}
AUL_Device_Close( board );
AUL_Terminate();
return SUCCESS;
ASIDE: The purpose of the library is to take care of much of the complexity and error checking between the kernel-module and the application relieving the end user from having to perform all of those operations. The library performs extensive checking for proper device (STX104 or Tracer) configurations. Further, the library is a way for Apex to provide ongoing maintenance and addition of new features without disrupting existing user application code. The library is linked into the user application code. The library will handle much of the activities that do not need to reside within the kernel-module (i.e. floating point operations).
Example #2:
STX104 board with base address 0x300 and Tracer board with base address 0x310 installed on a PC/104 stack.
Driver Installation:
sudo insmod mod_auldrv.ko offset=0x300,0x310 bytes=0x10,0x10 acronym="mystx104","mytracere" type="stx104","tracere"
Using the library we open the device as follows:
int board[2];
error_code = AUL_Initialization( NULL );
if ( error_code ) return error_code;
error_code = AUL_Device_Open( "mystx104", &(board[0]) );
if ( error_code )
{ printf( "Error code: %s\n", AUL_Last_Error_Get() );
AUL_Termination();
return error_code;
}
error_code = AUL_Device_Open( "mytracere", &(board[1]) );
if ( error_code )
{ printf( "Error code: %s\n", AUL_Last_Error_Get() );
AUL_Device_Close( board[1] );
AUL_Termination();
return error_code;
}
AUL_Device_Close( board[1] );
AUL_Device_Close( board[0] );
AUL_Terminate();
return SUCCESS;
HELPFUL LINUX COMMANDS
If you need more information you can use 'man' command in Linux on the command line to obtain more information or Google the particular command for further examples. This section has been added because these commands are essential for kernel driver installation, removal and debugging. There are additional examples uses following this list of commands.
| COMMAND | INFORMATION |
| dmesg | DESCRIPTION: Prints or controls the kernel ring buffer. SYNOPSIS: dmesg [--clear] [--console-on] [--console-off] [--console-level <level>] [--read-clear [options]] EXAMPLE: Refer to the dmesg example below. |
| modinfo | DESCRIPTION: A program to show information about a Linux Kernel module. SYNOPSIS: modinfo [-0] [-F field] [-k kernel] [modulename|filename ...] EXAMPLE: modinfo mod_auldrv |
| insmod | DESCRIPTION: A simple program to insert a module into the Linux Kernel. SYNOPSIS: insmod [filename] [module options ...] EXAMPLE: |
| rmmod | DESCRIPTION: A simple program to remove a module from the Linux Kernel SYNOPSIS: rmmod [-f] [-w] [-s] [-v] [modulename] EXAMPLE: rmmod mod_auldrv |
| lsmod | DESCRIPTION: A program to show the status of modules in the Linux Kernel. Use this command to verify that the mod_auldrv has actually been installed into the kernel after performing an insmod, for example. SYNOPSIS: lsmod EXAMPLE: lsmod |
| modprobe | DESCRIPTION: A program to show information about a Linux Kernel module. SYNOPSIS: modprobe [-v] [-V] [-C config-file] [-n] [-i] [-q] [-b] [modulename] [module parameters ...] modprobe [-r] [-v] [-n] [-i] [modulename ...] modprobe [-l] [-t dirname] [wildcard] modprobe [-c] modprobe [--dump-modversions] [filename] EXAMPLE: |
| uname | DESCRIPTION: Print system information. SYNOPSIS: uname [OPTION]... Print certain system information. With no OPTION, same as -s. -a, --all print all information, in the following order, except omit -p and -i if unknown: -s, --kernel-name print the kernel name -n, --nodename print the network node hostname -r, --kernel-release print the kernel release -v, --kernel-version print the kernel version -m, --machine print the machine hardware name -p, --processor print the processor type or "unknown" -i, --hardware-platform print the hardware platform or "unknown" -o, --operating-system print the operating system --help display this help and exit --version output version information and exit EXAMPLE: Print the kernel release information. uname -r |
More details regarding each of these commands can be obtained using the man command at a Linux terminal. For example, typing "man dmesg" will bring up a manual page regarding the dmesg command. Performing internet searches can reveal additional examples and insights. Additional, more length examples are illustrated next, especially for those commands that are particularly useful and are understated above.
Linux Command Example: dmesg
Below is a session illustrating the power of using the dmesg command. In this case we are executing the demo_basic application that is compiled in a debug mode. You can see how the kernel driver writes a series of kprintf() statements to the dmesg kernel buffer illustrating every communication with the kernel driver.
Clear the kernel ring buffer:
mike@Ubuntu32vb:~/aes_ws/demo_basic/_Out_Debug$ dmesg --clear dmesg: klogctl failed: Operation not permitted
Oops! We need to use "sudo" or "su" to make this command work:
mike@Ubuntu32vb:~/aes_ws/demo_basic/_Out_Debug$ sudo dmesg --clear [sudo] password for mike:
Run the demo_basic application to illustrate how the kernel driver writes debug information to the kernel ring buffer: mike@Ubuntu32vb:~/aes_ws/demo_basic/_Out_Debug$ ls demo_basic demo_basic.d demo_basic.o dmesg.txt makefile objects.mk sources.mk subdir.mk
mike@Ubuntu32vb:~/aes_ws/demo_basic/_Out_Debug$ ./demo_basic Library name = libauldebug.a AUL_Initialization successful Driver name = mod_auldrv AUL_Device_Open successful AUL_Device_Close successful AUL_Termination successful
Output the kernel ring buffer from which you can see all auldrv driver transactions! mike@Ubuntu32vb:~/aes_ws/demo_basic/_Out_Debug$ dmesg [69192.499315] auldrv AulDrv_Open entry. [69192.499327] auldrv AulDrv_Open: minor = 0, controller = auldrv [69192.499334] auldrv AulDrv_Open exit success. [69192.500001] auldrv AulDrv_Ioctl entry. [69192.500176] auldrv AulDrv_Ioctl exit success. [69192.500334] auldrv AulDrv_Ioctl entry. [69192.500342] auldrv AulDrv_Ioctl exit success. [69192.500458] auldrv AulDrv_Ioctl entry. [69192.500467] auldrv AulDrv_Ioctl_Device_Definition_Get entry. [69192.500474] auldrv AulDrv_Ioctl_Device_Definition_Get exit success. [69192.500481] auldrv AulDrv_Ioctl exit success. [69192.500730] auldrv AulDrv_Open entry. [69192.500740] auldrv AulDrv_Open: minor = 1, device = stx104 [69192.500747] DEBUG: IO_Open: entry [69192.500756] DEBUG: IO_LDGX86_Open: entry [69192.500765] DEBUG: IO_LDGX86_Open: exit success [69192.500773] DEBUG: IO_Open: exit success [69192.500781] auldrv AulDrv_Open exit success. [69192.500828] auldrv AulDrv_Ioctl entry. [69192.500836] DEBUG: IO_Ioctl: entry [69192.500844] DEBUG: IO_Xfr_Command: entry [69192.500858] DEBUG: IO_Read_U8: mem[0x030B]=>0xFF [69192.500866] DEBUG: IO_Xfr_Command: exit success [69192.500874] DEBUG: IO_Ioctl: exit success [69192.500882] auldrv AulDrv_Ioctl exit success. [69192.500890] auldrv AulDrv_Ioctl entry. [69192.500896] DEBUG: IO_Ioctl: entry [69192.500904] DEBUG: IO_Xfr_Command: entry [69192.500916] DEBUG: IO_Write_U8: mem[0x030E]<=0xF8 [69192.500924] DEBUG: IO_Xfr_Command: exit success [69192.500932] DEBUG: IO_Ioctl: exit success [69192.500939] auldrv AulDrv_Ioctl exit success. [69192.500946] auldrv AulDrv_Ioctl entry. [69192.500952] DEBUG: IO_Ioctl: entry [69192.500959] DEBUG: IO_Xfr_Command: entry [69192.500971] DEBUG: IO_Write_U16: mem[0x030C]<=0x55AA [69192.500979] DEBUG: IO_Xfr_Command: exit success [69192.500986] DEBUG: IO_Ioctl: exit success [69192.500993] auldrv AulDrv_Ioctl exit success. [69192.501000] auldrv AulDrv_Ioctl entry. [69192.501006] DEBUG: IO_Ioctl: entry [69192.501013] DEBUG: IO_Xfr_Command: entry [69192.501025] DEBUG: IO_Write_U8: mem[0x030E]<=0xF8 [69192.501032] DEBUG: IO_Xfr_Command: exit success [69192.501040] DEBUG: IO_Ioctl: exit success [69192.501047] auldrv AulDrv_Ioctl exit success. [69192.501054] auldrv AulDrv_Ioctl entry. [69192.501060] DEBUG: IO_Ioctl: entry [69192.501067] DEBUG: IO_Xfr_Command: entry [69192.501079] DEBUG: IO_Read_U16: mem[0x030C]=>0xFFFF [69192.501086] DEBUG: IO_Xfr_Command: exit success [69192.501094] DEBUG: IO_Ioctl: exit success [69192.501101] auldrv AulDrv_Ioctl exit success. [69192.501212] auldrv AulDrv_Close entry. [69192.501220] auldrv AulDrv_Close: minor = 1, device = stx104 [69192.501227] DEBUG: IO_Close: enter [69192.501234] DEBUG: IO_LDGX86_Close: enter [69192.501242] DEBUG: IO_LDGX86_Close: exit success [69192.501250] DEBUG: IO_Close: exit success [69192.501257] auldrv AulDrv_Close exit success. [69192.501633] auldrv AulDrv_Close entry. [69192.501642] auldrv AulDrv_Close: minor = 0, controller = auldrv [69192.501648] auldrv AulDrv_Close exit success. mike@Ubuntu32vb:~/v/demo_basic/_Out_Debug$
Linux Command Example: modinfo
Another very useful command is the modinfo. It is particularly useful in reminding one the parameters that can be passed into the mod_auldrv kernel module when using insmod command.
mike@Ubuntu32vb:~/aes_ws/drv_aul_x86$ modinfo mod_auldrv.ko
filename: mod_auldrv.ko
license: GPL
description: AES Generic Kernel Driver
author: customer.service@apexembeddedsystems.com
srcversion: E0C6949C860EA511FD001F2
depends:
vermagic: 3.2.0-75-generic-pae SMP mod_unload modversions 686
parm: major:This driver ID. Dynamic assignment when major=0 (default). (int)
parm: offset:region offset array. Ex: offset=0x300 or offset=0x300,0x310 (array of uint)
parm: bytes:region bytes array. Ex: bytes=0x10 or bytes=0x10,0x10 (array of uint)
parm: restrict:array of uint
parm: bit8:region restrict 8-bits. Ex: restrict=0 or restrict=0,1
parm: acronym:region name array. Ex: acronym="stx104" or acronym="stx104","tracer" (array of charp)
parm: type:type name array. Ex: type="sxt104" or type="stx104","tracer" (array of charp)