LTP(Linux Test Project)使用指南_ltp使用

1、ltp-ddt简介

1.1、ltp

LTP(Linux Test Project)是由SGI，OSDL和Bull发起的联合项目，由IBM，思科，富士通，SUSE，红帽，甲骨文等公司开发和维护。该项目的目标是向开源社区提供测试，以验证Linux的可靠性，健壮性和稳定性。

LTP测试套件包含一系列用于测试Linux内核和相关功能的工具。目标是通过将测试自动化带入测试工作来改进Linux内核和系统库。

从代码目录架构上看，LTP分为以下重要部分：

• 1、testcase目录:

该目录包含测试用例的具体实现文件(test logic files)，格式为：C代码、shell脚本。

ltp-ddt$ ls testcases/
commands  cve  ddt  kdump  kernel  lib  Makefile  misc  network  open_posix_testsuite  realtime
ltp-ddt$ ls testcases/kernel/
connectors  containers  controllers  device-drivers  firmware  fs  hotplug  include  input  io  ipc  lib  logging  Makefile  mem  module  numa  power_management  pty  sched  security  syscalls  timers  tracing
ltp-ddt$ ls testcases/kernel/syscalls/
abort         clock_nanosleep   execl       flock          getgroups        
accept        clock_nanosleep2  execle      fmtmsg         gethostbyname_r  
accept4       clone             execlp      fork           gethostid        
access        close             execv       fpathconf      gethostname      
acct          cma               execve      fstat          getitimer        
add_key       confstr           execvp      fstatat        get_mempolicy    
adjtimex      connect           exit        fstatfs        getpagesize      
alarm         copy_file_range   exit_group  fsync          getpeername          
...
1
2
3
4
5
6
7
8
9
10
11
12
13
14

• 2、runtest目录：

LTP测试框架不会直接调用testcase，而是通过一个中间包装脚本测试场景文件(test scenario files)来调用，放在runtest路径下面。一般一个testcase对应一个runtest文件。

ltp-ddt$ ls runtest/
admin_tools  controllers  dma_thread_diotest  fs_perms_simple  input        ltp-aiodio.part1      ltplite    
can          cpuhotplug   fcntl-locktests     fs_readonly      io           ltp-aiodio.part2      lvm.part1      
...  
ltp-ddt$ 
ltp-ddt$ cat runtest/syscalls
#DESCRIPTION:Kernel system calls
abort01 abort01

accept01 accept01
accept4_01 accept4_01

access01 access01
access02 access02
access03 access03
access04 access04
... 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

默认运行哪些runtest在scenario_groups/default文件中定义：

ltp-ddt$ ls scenario_groups/
default  default-ddt  Makefile  network
ltp-ddt$ 
ltp-ddt$ cat scenario_groups/default
syscalls
fs
fs_perms_simple
fsx
dio
io
...
1
2
3
4
5
6
7
8
9
10
11

• 3、runltp命令：

runltp是运行ltp测试的总入口命令。

编译、安装完ltp以后，使用以下命令运行：

$ ./runltp
1

也可以单独的运行某个testcase集：

$ ./runltp -f syscalls
1

也可以直接调用某个testcase的bin文件：

$ testcases/bin/fork13 -i 37
1

• 4、其他特性：

1.2、ltp-ddt

LTP-DDT是德州仪器（TI）用于验证Linux版本的测试应用程序。它基于LTP。

LTP验证许多内核区域，例如内存管理，调度程序和系统调用。LTP-DDT通过测试来扩展LTP的核心内核测试，以验证德州仪器开发的内核驱动程序。LTP-DDT专注于嵌入式设备驱动程序测试。它包含数百个验证设备驱动程序功能和性能的测试。LTP-DDT还包含用于验证系统用例和整体系统稳定性的测试。

LTP-DDT在LTP的基础之上进行了一系列的扩展：

• 1、testcase目录:

ltp-ddt在testcase目录当中增加了ddt目录：

ltp-ddt$ ls testcases/ddt/
alsa_test_suite  dmtimer_test_suite  fbdev_display_test_suite  gpio_test_suite  ipc_test_suite  rtc_test_suite  spidev_test_suite  utils                    v4l2_display_davinci_test_suite  wdt_test_suite
devfreq_drivers  edma_test_suite     filesystem_test_suite     i2c_test_suite   Makefile        scripts         usb_test_suite     v4l2_capture_test_suite  v4l2_display_test_suite
1
2
3

testcase除了支持C代码和shell脚本，还支持python脚本。python脚本放在ddt/scripts目录当中，以及对应的模板文件testcases/ddt/scripts/TEMPLATE：：

ltp-ddt$ ls testcases/ddt/scripts/
adc  alsa  blk  can  clk_fw  common  crypto  dma  dmtimer  drm  eth  gpio  graphics  i2c  ipc  Makefile  mtd  pci  powermgr  pwm  rng  rtc  scsi  TEMPLATE  thermal  touchscreen  uart  usb  video  wlan
1
2

• 2、runtest目录：

ltp-ddt在runtest目录当中增加了ddt目录：

ltp-ddt$ ls runtest/ddt/
adc_ti                                 gpio_rotaryencoder                         nand_mtd_dd_write_concurrence_ubifs     powermgr_suspend_emmc                    realtime_v4l2cap_load                   system_eth_smp_priority_processing
alsa_accesstype                        gpio-test-kerneldebug                      nand_mtd_debug_erase                    powermgr_suspend_ethernet                realtime_v4l2cap_load-smp               system_gpio_cpufreq
alsa_amixer_switchtoggle               gpio-tests                                 nand_mtd_debug_rw                       powermgr_suspend_graphics                remoteproc_r5f                          system_graphics_cpufreq
alsa_amixer_volumesetting              gst_capture                                nand_mtd_flash_erase                    powermgr_suspend_i2c                     rng_test                                system_gst_cpufreq
...
1
2
3
4
5
6

测试场景文件(test scenario files)增加了以下注释关键字：@name，@desc，@requires和@setup_requires。
@requires和@setup_requires用于根据PLATFORM功能在运行时选择测试用例。

测试场景文件(test scenario files)具体的格式如下：

 // 正式部分，每一行对应一个测试步骤，一个测试步骤的格式为：<TAG> <COMMANDS> 
 * Follow LTP guidelines. The test scenario file is basically made of one or 
   more test step lines. Each test step line have following format
   <TAG> <COMMANDS>, where
   TAG is a string that identifies the test step.
     Use following convention to named TAGs so that the test cases can be 
     selectively run based on AREA, SCOPE and/or TYPE.  
     <AREA>_<SCOPE>_<TYPE>_<OPT_ID>,
      i.e. “NAND_S_FUNC_RW_8K”, “NAND_M_PERF_ALL-SIZES” 
     The SCOPE tags are used to indicate the amount of time require to run
     the tests, giving users ability to filter test cases based on estimated
     execution time.
     SCOPE TAGS:
      'XS', 'S', 'M', 'L', 'XL', 'XXL' (for eXtra Small, Small, Medium, etc.
		     Just imagine you are buying clothes ;)
      We used the following rough guidelines to determine test duration based 
      on scope:
        XS:  <= 1  min
        S:   <= 10 mins
        M:   <= 1  hour
        L:   <= 8  hours
        XL:  <= 24 hours
        XXL: <= 1  week
     TYPE TAGS:
      ‘FUNC’, ‘PERF’, ‘STRESS’, ‘USECASE’, ‘COMPLIANCE’, ‘MODULAR’, ‘DOC’
   COMMANDS is a list of one or more shell commands separated by semicolon (;),
     the test step will pass if the commands return zero, otherwise it fails.

 // 默认的模板文件为runtest/ddt/TEMPLATE
 * Use the default test scenario file template available at
   runtest/ddt/TEMPLATE as a starting point to develop your test scenario.

 // 注释部分，可以使用@requires关键字来限定测试需要的ARCH, DRIVER, SOC and/or MACHINE等条件。
 * Use the @requires annotation to specify ARCH, DRIVER, SOC and/or MACHINE 
   requirements to run the test scenario. You can use (), &&, ||, * to specify
   the test requirements. Examples:
   
   @requires /net/eth/* && spi_master
   To run this test the platform must have an ethernet driver and a 
   spi_master driver

   @requires am3517-evm
   This test can only be run on an am3517 EVM.

   @requires (mmc_host || nand) && armv*
   This test requires mmc or nand drivers and an ARM architecture

 // 注释部分，可以使用@setup_requires来描述测试需要的外部条件，比如连接外部usb设备等等。
 * Use the @setup_requires annotation to specify test setup requirements.
   Some test cases like USB and Video capture requires special peripherals, such as 
   USB flash drives, DVD players, video cameras, etc., to be connected to the DUT.
   Using @setup_requires the test developer highlights such test setup requirements. 
   This information might be used by test automation frameworks to allocate test requests
   to DUTs that have the appropriate peripherals connected to them.
   Please follow the naming conventions identified in section 9) of this document
   if the test scenario needs to identify any setup requirements.

   You can use underscore(_) to seperate multiple @setup_requires. Examples:

   @setup_requires usbhostvideo_usbhostaudio
   This test requires usbhostvideo setup and usbhostaudio setup.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

ltp-ddt对应新增了默认运行文件scenario_groups/default-ddt：

ltp-ddt$ ls scenario_groups/
default  default-ddt  Makefile  network
ltp-ddt$ 
ltp-ddt$ cat scenario_groups/default-ddt 
ddt/alsa_samplerate
ddt/clcd
ddt/edma
ddt/edma_chain
ddt/edma_link
ddt/emmc_quick_perf
...
1
2
3
4
5
6
7
8
9
10
11

• 3、platforms目录：

ltp-ddt完全新增了一个文件夹platforms/：

ltp-ddt$ ls platforms/
am170x-evm  am335x-hsevm  am37x-evm   am437x-sk     am571x-idk    am57xx-beagle-x15  am654x-idk        da830-omapl137-evm  dm385-evm   dra71x-hsevm  dra76x-evm       hikey      k2g-hsevm   k2l-evm    omap5-evm      tci6614-evm
...
ltp-ddt$ cat platforms/am335x-evm 
armv7l
am335x
am335x-evm
adc/tscadc
can
crypto/crypto-omap
...
1
2
3
4
5
6
7
8
9
10
11

该文件根据具体平台设置了一组配置，在测试的时候可以使用这组配置去挑选当前平台能支持的testcase。

平台配置文件的格式如下：

 // 默认的模板文件：platforms/TEMPLATE
 * Copy the default platform file available at platforms/TEMPLATE to 
   platforms/<your platform>. <your platform> name is typically the evm name

 // 在编写ltp-ddt脚本时，请使用以下的平台文件名称与$ MACHINE进行比较
 * Please use the following names for platform files and to compare against 
   $MACHINE when writing ltp-ddt scripts:

    am180x-evm   arago-armv7         dm355-evm    dm814x-evm
    am181x-evm   dm365-evm           dm816x-evm
    am3517-evm   beagleboard         dm368-evm    
    am37x-evm    c6a814x-evm         dm37x-evm    omap3evm
    am387x-evm   c6a816x-evm         dm6446-evm   tnetv107x-evm
    am389x-evm   da830-omapl137-evm  dm6467-evm   am335x-evm
    arago-armv5  da850-omapl138-evm  dm6467t-evm  beaglebone

 // 文件格式：
 // 平台文件前3行必须是： architecture, SoC and EVM
 // 后面每一行对应一个驱动，驱动名符合`/sys/class`文件层次下的命名
 * Modify your platform file based on the capabilities supported by the new evm
   The platform file identifies the architecture, the SoC, the evm and the
   supported drivers. The supported drivers lines follow a variation of the
   hierarchy used in /sys/class but it is not exactly the same. Hence it is 
   important to use the platforms/TEMPLATE file as your starting point.
   Please note the first 3 lines of the platform file MUST identify, the 
   architecture, SoC and EVM respectively, follow by one or more driver lines.
   Typically the architecture and machine name used in the platform file are
   the ones reported by uname -a.
   Sample platform file:
    armv7l
    am3517
    am3517-evm
    net/eth/davinci_emac
    nand/omap2-nand
    ehci/ehci-omap
    i2c-adapter/i2c_omap
    mmc_host/mmci-omap-hs
    rtc/rtc-s35390a
    watchdog/omap_wdt
    ...

 * You might need to define new override values for your new platform in some
   test case files (see section 4.1 above for details). A reasonable strategy
   is to try to run an existing test plan and then analyze the test failures
   to determine probable test cases where you need to define override values.  

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

• 4、runltp命令：

ltp-ddt的runltp命令可以使用平台文件来定义需要运行的testcase。

运行制定平台的所有testcase：

./runltp -P am335x-evm
1

运行指定平台的指定testcase：

./runltp -P am335x-evm -f ddt/lmbench
1

2、环境构造

2.1、交叉编译

ltp-ddt的交叉编译：

linux-3.2.0$ make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm headers_install
  CHK     include/linux/version.h
  INSTALL include/linux/mmc (1 file)
  INSTALL include/linux/netfilter/ipset (4 files)
  INSTALL include/linux/netfilter (69 files)
  INSTALL include/linux/netfilter_arp (2 files)
  INSTALL include/linux/netfilter_bridge (18 files)
  INSTALL include/linux/netfilter_ipv4 (15 files)
  INSTALL include/linux/netfilter_ipv6 (11 files)
  INSTALL include/linux/nfsd (4 files)
  INSTALL include/linux/raid (2 files)
  INSTALL include/linux/spi (1 file)
  INSTALL include/linux/sunrpc (1 file)
  INSTALL include/linux/tc_act (7 files)
  INSTALL include/linux/tc_ematch (4 files)
  INSTALL include/linux/usb (10 files)
  INSTALL include/linux/wimax (1 file)
  INSTALL include/linux (366 files)
  INSTALL include/mtd (5 files)
  INSTALL include/rdma (6 files)
  INSTALL include/scsi/fc (4 files)
  INSTALL include/scsi (3 files)
  INSTALL include/sound (8 files)
  INSTALL include/video (3 files)
  INSTALL include/xen (2 files)
  INSTALL include (0 file)
  INSTALL include/asm (32 files)
linux-3.2.0$ pwd
~/linux-3.2.0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

ltp-ddt$ make autotools
sed -n '1{s:LTP-:m4_define([LTP_VERSION],[:;s:$:]):;p;q}' VERSION > m4/ltp-version.m4
aclocal -I m4
autoconf
autoheader
automake -c -a
configure.ac:20: installing './compile'
configure.ac:18: installing './config.guess'
configure.ac:18: installing './config.sub'
configure.ac:4: installing './install-sh'
configure.ac:4: installing './missing'
make -C testcases/realtime autotools
make[1]: 正在进入目录 `~/ltp-ddt/testcases/realtime'
aclocal -I ~/ltp-ddt/testcases/realtime/m4
autoconf
autoheader
autoheader
automake -c -a
configure.ac:9: installing './compile'
configure.ac:14: installing './config.guess'
configure.ac:14: installing './config.sub'
configure.ac:4: installing './install-sh'
configure.ac:4: installing './missing'
make[1]:正在离开目录 `~/ltp-ddt/testcases/realtime'
ltp-ddt$
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

ltp-ddt$ export CROSS_COMPILER=arm-linux-gnueabihf-
ltp-ddt$ export CC=${CROSS_COMPILER}gcc
ltp-ddt$ export LD=${CROSS_COMPILER}ld
ltp-ddt$ export AR=${CROSS_COMPILER}ar
ltp-ddt$ export STRIP=${CROSS_COMPILER}strip
ltp-ddt$ export RANLIB=${CROSS_COMPILER}ranlib
1
2
3
4
5
6

./configure --host=arm-linux-gnueabihf

./configure --host=arm-linux-gnueabihf CC=arm-linux-gnueabihf-gcc LD=arm-linux-gnueabihf-ld AR=arm-linux-gnueabihf-ar STRIP=arm-linux-gnueabihf-strip RANLIB=arm-linux-gnueabihf-ranlib
1
2
3

ltp-ddt$ make SKIP_IDCHECK=1 KERNEL_USR_INC=~/linux-3.2.0/usr/include/ CROSS_COMPILE=arm-linux-gnueabihf- clean
ltp-ddt$ make SKIP_IDCHECK=1 KERNEL_USR_INC=~/linux-3.2.0/usr/include/ CROSS_COMPILE=arm-linux-gnueabihf-
1
2

make DESTDIR=~/rootfs_yaff2/home/root/ltp-ddt-install SKIP_IDCHECK=1 PLATFORM=am335x-evm install
1

ltp的交叉编译过程和ltp-ddt一样。

2.2、文件系统

sudo mkyaffs2image rootfs_yaff2/ ubi.img
1

交叉编译sysstat：

./configure --host=arm-linux-gnueabihf --cache-file=~/sysstat-12.0.5/cache_file_0  --prefix=~/sysstat_install/ --exec-prefix=~/sysstat_install/
make
make install
1
2
3

交叉编译busybox：

1、配置
$ make menuconfig

1.1、静态编译：
Build Options --->
通过空格键使能Build BusyBox as a static binary（no shared libs）（对应的方括号处显示星号*即为使能状态）。

1.2、设置交叉工具链：
Build Options --->
设置Cross Compiler prefix如下所示（具体路径和交叉工具链前缀根据实际情况修改）：
/usr/bin/arm-linux-gnueabihf-

1.3、安装路径设置：
Busybox Settings --->
    Installation Options("make install" behavior) --->
输入安装路径即可：
~/busybox_install/

1.4、不包含/usr目录：
Busybox Settings --->
    General Configuration --->
        [*]Don't use /usr
这个必需选中，否则在安装的时候将会将BusyBox安装在Ubuntu的/usr目录中，从而损坏了宿主机的根文件系统！

2、编译&安装
$ make
$ make install

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

3、测试运行

./runltp -P am335x-evm -f syscalls
./runltp -P am335x-evm -f ddt/lmbench
./runltp -P am335x-evm -f nandtest
./runltp -P am335x-evm -f uart
1
2
3
4

example: runltp -c 2 -i 2 -m 2,4,10240,1 -D 2,10,10240,1 -p -q  -l /tmp/result-log.1879 -o /tmp/result-output.1879 -C /tmp/result-failed.1879 -d /home/root/sda1/ltp-ddt-install/opt/ltp
1