The Open Group : Making Standards Work
About The Open Group The Open Group Forums Certification Collaboration Services
HOME   |   SITE MAP  
Sponsor an Event Become a Member of The Open Group The Open Group Member Area
Government Programs Events Bookstore & Downloads Newsroom Contact The Open Group
You are here: Home >Testing >Test Suites > VSRT Overview
Test Suites
 
VSRT

POSIX Realtime Test Suite (for former 1003.1b, now the Realtime option in 1003.1-200x)

Also covers former 1003.1d, 1003.1j and 1003.1q standards - Advanced Realtime, Realtime Extensions and Tracing.

Summary

  • The industry standard test suite with test coverage for the IEEE POSIX® Realtime extension.

  • Verifies the POSIX 1003.1b-1993/1003.1i-1995 Realtime extension and the UNIX 98 _XOPEN_REALTIME Feature Group.

  • In the Westwood version additionally verifies the POSIX 1003.1, 2003 Edition the UNIX 03 _XOPEN_REALTIME and _XOPEN_ADVANCED_REALTIME Feature Groups.

  • Built upon the proven VSXgen harness, yielding a configurable and productive tool.

  • Using the VSXgen architecture, this suite is provided as a VSXgen testset.

  • Coverage for functions and header file tests.

  • Test modes for UNIX 98 and POSIX Realtime

  • In the Westwood version Test modes for UNIX 03 and POSIX.1, 2003 edition

  • Assertions following the IEEE Std 1003.3 have been developed for all tests, greatly aiding failure analysis.

  • Optional for testing the Single UNIX® Specification Version 2 and 3.

  • The VSRT-PSE derivative provides an embedded test capability

  • Developed, owned and supported by The Open Group.

If your organization is developing an implementation conforming to the IEEE POSIX Realtime standards then this test suite gives significant test coverage and is recommended for in-house quality-assurance as well as part of your branding campaign.

Keywords: POSIX® Realtime, 1003.1b, UNIX® 98, UNI® 03, POSIX 1003.1,2003 Edition Realtime, Testing, POSIX, X/Open.

Top



Introduction

The Open Group's Realtime Test Suite, known as VSRT, is a stand alone test suite covering POSIX 1003.1b, 1003.1d, 1003.1j, 1003.1q and their associated Technical Corrigenda. The most recent update also includes test modes for the UNIX 03 and IEEE Std 1003.1, 2003 Edition

VSRT uses the robust VSXgen test harness, which in turn uses the industry standard Test Environment Toolkit (TET3) . The benefits of using VSXgen, which is derived from VSX4, is that the harness is familiar to existing users of The Open Group's Test suites, it has proven portability and robustness characteristics and it allows the VSRT tests to be integrated with other testsets that run under VSXgen, making the test suites easily useable for in-house quality assurance and automated regresssion testing.

TET3 is the latest version of the Test Environment Toolkit and supports multi-threaded testing. More information on the Test Environment Toolkit is described on the TETworks web site.

Top



Test Coverage

The test suite covers the following functional areas in the IEEE POSIX 1003.1b/1003.1d/1003.1j/1003.1q standards, IEEE Std 1003.1, 2003 Edition/ UNIX 98/ UNIX 03.

POSIX 1003.1b Test Extension

  • Realtime Signals
  • Tested by generating series of realtime signals to verify queueing, reliability, prioritization, and differentiation between mutiple occurrences.

  • Synchronized input and output
  • Tested by generating I/O and verifying it is appropriately synchronized on request. As synchronization can occur independent of this API, this testing will be less than completely comprehensive.

  • Asynchronized input and output
  • Tested by generating series of asynchronous I/O requests and verifying queueing, logical parallelism, signal generation, event cancellation, event waiting, etc.

  • Semaphores
  • Tested by generating multiple processes and verifying they can correctly synchronize operation with semaphores under various conditions.

  • Memory locking
  • Tested by locking and unlocking process memory. As the effects of locking are difficult to discern, this testing will be less than comprehensive.

  • Memory mapped files and shared memory
  • Tested by creating and manipulating multiple shared memory objects and mapped files.

  • Priority scheduling
  • Tested using multiple processes to verify SCHED_FIFO and SCHED_RR scheduling algorithms operate as specified. SCHED_OTHER will also be verified if identical to SCHED_FIFO or SCHED_RR. The timing and ordering of process scheduling will be verified for the defined algorithms and parameters.

  • High resolution clocks and timers
  • Tested by creating and manipulating various timers to verify granularity, accuracy, signal generation, etc.

  • IPC Message passing
  • Tested by generating multiple processes and verifying they can correctly communicate messages under various conditions.


The POSIX Realtime functions tested are:

aio_cancel() aio_error() aio_fsync() aio_read()
aio_return() aio_suspend() aio_write()

clock_getres() clock_gettime() clock_settime()

fdatasync()

lio_listio()

fchmod()
mlock() mlockall()
mmap() munmap() mprotect()
mq_close() mq_getattr() mq_notify() mq_open()
mq_receive() mq_send() mq_setattr() mq_unlink()
ftruncate()
msync() fsync()
munlock() munlockall()

nanosleep()

sched_get_priority_max() sched_get_priority_min() sched_getparam()
sched_getscheduler() sched_rr_get_interval() sched_setparam()
sched_setscheduler() sched_yield()
sem_close() sem_destroy() sem_getvalue() sem_init()
sem_open() sem_post() sem_trywait() sem_unlink() sem_wait()
shm_open() shm_unlink()
sigqueue() sigtimedwait() sigwaitinfo()

timer_create() timer_delete() timer_getoverrun() timer_gettime()
timer_settime()

The headers to be tested are:

<aio.h> <mqueue.h> <sched.h> <sys/mman.h>

POSIX 1003.1j Test Extension


Functional Areas Tested

The test suite covers the following functional areas in the IEEE POSIX 1003.1j standard.

Memory management

A facility to allow programs to allocate or access different kinds of physical memory that are present in the system, and allow separate application programs to share portions of this memory.

Synchronization

Synchronization primitives that allow multiprocessor applications to achieve the performance benefits of their hardware architecture.

Clocks and Timers

The addition of the Monotonic Clock, the specification of the effects of setting the time of a clock on other timing services, and the addition of functions to support relative or absolute suspension based upon a clock specified by the application.


POSIX Functions Tested

The following new POSIX functions are tested:


posix_typed_mem_open()
posix_mem_offset ()
posix_typed_mem_get_info()
pthread_spin_init()
pthread_spin_destroy()
pthread_spin_lock()
pthread_spin_trylock()
pthread_spin_unlock()
pthread_barrierattr_init()
pthread_barrierattr_destroy()
pthread_barrierattr_getpshared()
pthread_barrierattr_setpshared()
pthread_barrier_init()
pthread_barrier_destroy()
pthread_barrier_wait()
pthread_rwlockattr_init()
pthread_rwlockattr_destroy()
pthread_rwlockattr_getpshared()
pthread_rwlockattr_setpshared()
pthread_rwlock_init()
pthread_rwlock_destroy()
pthread_rwlock_rdlock()
pthread_rwlock_tryrdlock()
pthread_rwlock_timedrdlock()
pthread_rwlock_wrlock()
pthread_rwlock_trywrlock()
pthread_rwlock_timedwrlock()
pthread_rwlock_unlock()
clock_nanosleep()
pthread_condattr_setclock()
pthread_condattr_getclock()

Tests are also performed on existing functions POSIX .1 functions whose behaviour is modified by the advanced realtime functionality.


Header Files Tested

The following header files are - tested:


<sys/mman.h>
<pthread.h>
<stat.h>
<time.h>
<sys/types.h>
<unistd.h>

POSIX 1003.1d Test Extension


Functional Areas Tested

The test suite covers the following functional areas in the IEEE POSIX 1003.1d standard.

Spawn a Process

New system services to spawn the execution of a new process in an efficient manner.

Timeouts for some blocking services

Additional services that provide a timeout capability to system services already defined in POSIX.1b and POSIX.1c, thus allowing the application to include better error detection and recovery capabilities.

Sporadic Server Scheduling

The addition of a new scheduling policy appropriate for scheduling aperiodic processes or threads in hard real-time applications.

Execution Time Clocks and Timers

The addition of new clocks that measure the execution times of processes or threads, and the possibility to create timers based upon these clocks, for runtime detection (and treatment) of execution time overruns.

Advisory Information for File Management

Addition of services that allow the application to specify advisory information that can be used by the system to achieve better or even deterministic response times in file management or input & output operations.


POSIX Functions Tested

The following new POSIX functions are tested:


posix_spawn()
posix_spawn_file_actions_addclose()
posix_spawn_file_actions_adddup2()
posix_spawn_file_actions_addopen()
posix_spawn_file_actions_destroy()
posix_spawn_file_actions_init()
posix_spawnattr_destroy()
posix_spawnattr_getflags()
posix_spawnattr_getpgroup()
posix_spawnattr_getschedparam()
posix_spawnattr_getschedpolicy()
posix_spawnattr_getsigdefault()
posix_spawnattr_getsigmask()
posix_spawnattr_init ()
posix_spawnattr_setflags()
posix_spawnattr_setpgroup()
posix_spawnattr_setschedparam()
posix_spawnattr_setschedpolicy()
posix_spawnattr_setsigdefault()
posix_spawnattr_setsigmask()
posix_spawnp()
pthread_mutex_timedlock()
sem_timedwait()
mq_timedreceive()
mq_timedsend()
clock_getcpuclockid()
pthread_getcpuclockid()
posix_fadvise()
posix_fallocate()
posix_madvise()
posix_memalign()

Tests are also performed on existing functions POSIX .1 functions whose behaviour is modified by the additional realtime extensions.


Header Files Tested

The following header files are tested:


<fcntl.h>
<limits.h>
<sys/mman.h>
<mqueue.h>
<pthread.h>
<sched.h>
<semaphore.h>
<spawn.h>
<stdlib.h>
<time.h>
<unistd.h>

POSIX 1003.1q Test Extension

Functional Areas Tested

The test suite covers the following functional areas in the IEEE POSIX 1003.1q standard.


POSIX Functions Tested

The following new POSIX functions are tested:


pthread_attr_gettracingstate()
pthread_attr_settracingstate()
pthread_gettracingstate()
pthread_settracingstate()
posix_trace_attr_destroy()
posix_trace_attr_getclockres()
posix_trace_attr_getcreatetime()
posix_trace_attr_getgenversion()
posix_trace_attr_getinherited()
posix_trace_attr_getlogfullpolicy()
posix_trace_attr_getmaxdatasize()
posix_trace_attr_getlogsize()
posix_trace_attr_getstreamsize()
posix_trace_attr_getmaxsystemeventsize()
posix_trace_attr_getname()
posix_trace_attr_getstreamfullpolicy()
posix_trace_attr_getusereventsize()
posix_trace_attr_getutsname()
posix_trace_attr_init()
posix_trace_attr_setinherited()
posix_trace_attr_setlogfullpolicy()
posix_trace_attr_setmaxdatasize()
posix_trace_attr_setlogsize()
posix_trace_attr_setstreamsize()
posix_trace_attr_setname()
posix_trace_attr_setstreamfullpolicy()
posix_trace_close()
posix_trace_create()
posix_trace_create_withlog()
posix_trace_eventid_open()
posix_trace_eventid_equal()
posix_trace_trid_eventid_open()
posix_trace_eventid_get_name()
posix_trace_eventtypelist_getnext_id()
posix_trace_eventtypelist_rewind()
posix_trace_eventset_add()
posix_trace_eventset_del()
posix_trace_eventset_empty()
posix_trace_eventset_fill()
posix_trace_eventset_complement()
posix_trace_eventset_ismember()
posix_trace_flush()
posix_trace_get_attr()
posix_trace_get_filter()
posix_trace_get_status()
posix_trace_getnext_event()
posix_trace_open()
posix_trace_rewind()
posix_trace_set_filter()
posix_trace_shutdown()
posix_trace_start()
posix_trace_stop()
posix_trace_clear()
posix_trace_timedgetnext_event()
posix_trace_trygetnext_event()
posix_trace_event()

Tests are also performed on existing functions POSIX .1 functions whose behaviour is modified by the trace functionality.


Header Files Tested

The following header files are tested:

<pthread.h>
<trace.h>

Top



Modular Design

VSRT is modular, both at the implementation level and at the user level. The testing process is divided into clear stages -

Each stage is clearly documented in the User and Installation Guide, which accompanies the product.

Preparation

This stage includes setting up user-ids and groups for testing, and reading the tape. This stage includes configuring and installing the TET harness, which is used to control building, execution and cleanup of VSRT tests.

Configuration and Installation

VSRT automatically installs on most common platforms. Where system dependencies may vary, these are isolated into clearly defined user changeable sections of the test suite. During this stage VSRT builds the test harness and associated libraries.

Test Building

VSRT separates the building of the test harness and tools from the testsets themselves. The test suite may be built completely or in sub-sections, which decrease in granularity from test sections and areas down to individual testsets.

Test Execution

Once built, the test suite may be run as a whole in one pass, or in sub-sections which decrease in granularity from test areas down to individual tests within testsets. If you are producing a branding application the test suite must run in a single uninterrupted pass.

Test Reporting

VSRT generates an intermediate level journal file in a TET compliant format. This is sufficient for debugging purposes. When a formal report is required, such as for The Open Brand then the formal report writers provided with VSRT are used to process the journal file.

Top


POSIX is a registered trademark of the IEEE

Further Information

Top

   
 

 

 

 
   |   Legal Notices & Terms of Use   |   Privacy Statement   |   Top of Page   Return to Top of Page