Memory Spaces

This module provides a concurrent-safe interface to VxWorks memory spaces. It also range-checks the code at compile time to make sure you're not accessing hardware beyond its address range. And lastly, it generates code that is as efficient as hand-written assembly.

Rich Neswold gave a talk about this API using these slides.

NOTE: Starting with v2.7, a nested, version namespace was introduced to enforce matching APIs when building and deploying. In the following code examples, the namespace is referred to as "VWPP". This translates to vwpp for pre-2.7 libraries and to vwpp::v2_7 for v2.7.

VME Memory Class

This section covers the API for the class that handles VME memory spaces. This template class allows a region of memory to be defined with constant traits that can be used at compile-time to detect and prevent improper use. For this module to work correctly, your VxWorks kernel must have a working sysBusToLocalAdrs().

`VME::Memory` API

Constructor

template <VME::AddressSpace tag, VME::DataAccess DA, size_t size, typename LockType>
class VME::Memory(uint32_t offset)

Creates an object that manages a range of VME memory space. The constructor parameter, offset, defines the hardware's base offset in the memory space. The first template parameter, tag, specifies which type of VME memory is to be accessed: VWPP::VME::A16, VWPP::VME::A24, or VWPP::VME::A32. The DA parameter specifies which data widths can be accessed atomically. The value are VME::D8, VME::D16, VME::D8_D16, VME::D32, VME::D8_D32, VME::D16_D32, or VME::D8_D16_D32. The template parameter size defines how much address space is managed by the object. The compiler will guarantee, at compile time, no accesses will be made outside of this space. The last template parameter, LockType, is the type of lock that needs to be provided when accessing hardware.

A partially specialized version of the template is provided: if LockType is void, the same API is generated with the Lock parameters removed. This allows an implementer to use the compile-time range checking while letting higher-up code handle the serialization.

Copy Constructor

template <VME::AddressSpace tag, VME::DataAccess DA, size_t size, typename LockType>
template <typename NewLockType>
class VME::Memory(Memory<tag, DA, size, NewLockType> const& o)

The default copy constructor is not available (memory spaces should be passed by reference.) But there are rare situations in which it's useful to use a different lock for a memory space. This templated copy constructor allows you to make a copy of the memory space object, but use a different lock. For instance, a driver that has nested objects may find it difficult to pass the lock object through the constructors. Since constructors are single-threaded, it's safe to ignore serialization primitives. This template can be used to convert a memory space to have a void lock type. This feature should be used sparingly since it can break atomicity or even cause deadlocks if different mutexes are being used to serialize access. It might be better to specify a VWPP::IntLock so that interrupts are off while the hardware is initialized.

Take this example:

 1 using namespace VWPP;
 2
 3 template <typename LockType>
 4 class Card : private Uncopyable {
 5     typedef typename VME::Memory<VME::A16, VME::D8_D16, 32, LockType> HW;
 6
 7     HW hw;
 8
 9  public:
10     Card(uint32_t offset) : hw(offset)
11     {
12         typedef typename VME::Memory<VME::A16, VME::D8_D16, 32, void> Init_HW;
13
14         Init_HW ihw(hw);
15
16         // Use ihw
17     }
18 };

(Line 3) This driver is designed to be used with other drivers, so it lets the programmer specify the lock type in the template parameter.
(Line 5) Defines a type, HW, which is a VME A16 memory space of 32 bytes. Access to the memory will be serialized with the lock type given by the class template parameter.
(Line 12) Inside the constructor, we need to access the hardware. There's no guarantee, however, that we can create a lock to access the hardware; locks to global mutexes can be made but locks to mutexes in objects require a reference to the object, which we don't have. Since construction of an object (even nested objects) is a single-threaded operation, we can create a copy of the hardware object but set the lock type to void which removes the locking requirement from the API.
(Line 14) This creates the copy.
(Line 16) At this point, the hardware can be accessed through the new object and doesn't require a lock. The same approach can be used in the destructor.

Future enhancement

This constructor should allow the newly created memory object to be a subset of the source. For instance, an Industry Pack carrier driver would define a memory bank of 256 bytes for each slot and pass it to the driver's constructor. The driver would know whether the hardware uses a subset of the 256-bytes and can define a smaller bank of memory.

template <typename RegTrait>
RegTrait::Type VME::Memory::get() const

template <typename RegTrait>
RegTrait::Type VME::Memory::get(LockType const&) const

Returns a value of type RegTrait::Type from memory. Use the VME::Register template to define a register's traits. If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. Use of this method lets the compiler -- at compile time -- verify the entire access is within the memory space, the memory supports the data width, and whether reading is allowed.

template <typename RegTrait>
RegTrait::Type VME::Memory::get_element(size_t idx) const

template <typename RegTrait>
RegTrait::Type VME::Memory::get_element(LockType const&, size_t idx) const

Returns a value of type RegTrait::Type from a register in an array. Use the VME::Register template to define a register's traits (which at the very least must have a type in the form T[N].) If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. Use of this method lets the compiler -- at compile time -- verify the memory supports the data width, and whether reading is allowed.

template <typename RegTrait>
void VME::Memory::set(RegTrait::Type const& v) const

template <typename RegTrait>
void VME::Memory::set(LockType const&, RegTrait::Type const& v) const

Sets the location in memory, specified by the register traits, to the value v. Use the VME::Register template to define a register's traits. If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. Use of this method lets the compiler -- at compile time -- verify the entire access is within the memory space, the memory supports the data width, and whether writing is allowed.

template <typename RegTrait>
void VME::Memory::set_element(size_t idx, RegTrait::Type const& v) const

template <typename RegTrait>
void VME::Memory::set_element(LockType const&, size_t idx, RegTrait::Type const& v) const

Sets a register's value (of an array of registers) to the value v. Use the VME::Register template to define a register's traits. If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. Use of this method lets the compiler -- at compile time -- verify the memory supports the data width, and whether writing is allowed.

template <typename RegTrait>
void VME::Memory::set_field(RegTrait::Type const& mask, RegTrait::Type const& v) const

template <typename RegTrait>
void VME::Memory::set_field(LockType const&, RegTrait::Type const& mask, RegTrait::Type const& v) const

Does a read and set to a location in memory specified by the register trait. Use the VME::Register template to define a register's traits. The location is read and modified with the value, v. The mask parameter specifies which bits can be modified. If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. Use of this method lets the compiler -- at compile time -- verify the entire access is within the memory space, the memory supports the data width, and whether writing is allowed.

This method is only available to ReadAccess::Read registers.

This method is a little more efficient than using get<>() and set<>() to to update fields in a register. It's also more readable:

mem.set_field<Reg>(lock, 0xcc, 0x77);

verses

mem.set<Reg>(lock, (mem.get<regC>(lock) & ~0xcc) | (0xcc & 0x77));

template <typename T>
T VME::Memory::unsafe_get(size_t offset) const

template <typename T>
T VME::Memory::unsafe_get(LockType const&, size_t offset) const

Returns a value of type T from memory. If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. This method is "unsafe" because it doesn't check whether the location resides in the memory space or whether the hardware supports the size of the data access. Using this method will also require the use of one of the synchronization macros, MEMORY_SYNC, INSTRUCTION_SYNC, or ALL_SYNC, for correct operation.

template <typename T>
void VME::Memory::unsafe_set(size_t offset, T const& v) const

template <typename T>
void VME::Memory::unsafe_set(LockType const&, size_t offset, T const& v) const

Sets a T-sized location in memory to the value, v. The offset into the memory space is given by the offset parameter. If the memory bank requires locking, this function requires an appropriate LockType to be passed to prove you have exclusive access to the hardware. This method is "unsafe" because it doesn't check whether the location resides in the memory space or whether the hardware supports the size of the data access. Using this method will also require the use of one of the synchronization macros, MEMORY_SYNC, INSTRUCTION_SYNC, or ALL_SYNC, for correct operation.

`VME::Register` Class

This template class allows a hardware register to be defined with certain "traits".

`VME::Register` API

template <VME::AddressSpace tag, typename T, size_t Offset, VME::ReadAccess RA, VME::WriteAccess WA>
struct Register

This template is used to declare the traits of a hardware register. The template parameters determine the following:

AddressSpace tag specifies in which address space the register resides.
typename T specifies the size of the register. The library has only been tested with uint8_t, uint16_t, and uint32_t types. Hardware registers that match other C++ types are unusual. Officially, behavior is undefined if you use a non-integral data type.
size_t Offset specifies the offset of the register from the hardware's base address.
VME::ReadAccess RA specifies how to read the register. Any value in the VME::ReadAccess enumeration can be used. NoRead means you can only write to the register; attempting to read it with VME::Memory::get<>() will result in a compile-time error. Read means the register can be read. The generated code will guarantee that all previous reads and writes to memory complete before reading the register. DestructiveRead indicates that reading the register changes the state of the hardware. The generated code will prevent access to this register until all previous instructions have completed.
VME::WriteAccess WA specifies how to write to the register. Any value in the VME::WriteAccess enumeration can be used. @NoWrite@ means the register is read-only. You will get a compile-time error trying to write to it. Write means the register is writable. ConfirmWrite guarantees the write has reached the hardware before continuing.

As an example, let's say we have some hardware with two 16-bit registers. At offset 0, there's a read-only status register which, among other things, can tell us if there's something in a FIFO. At offset 2 is a 16-bit register which, when read, returns the oldest value in the FIFO. Since reading the FIFO removes the value, it's a destructive read so we need to use DestructiveRead. We should define the hardware registers as:

typedef VME:Register<A16, uint16_t, 0, Read, NoWrite> regStatus;
typedef VME:Register<A16, uint16_t, 0, DestructiveRead, NoWrite> regFifo;

Now we can write a loop that processes item in the FIFO until empty:

VMW::Memory<...> mem;

while (mem.get<regStatus>()) {
    uint16_t value = mem.get<regFifo>();

    // Do stuff with the value
}

// Nothing left in FIFO.

The classes in this library generate code that properly protects us from the CPU's instruction prefetch from reading the FIFO register before the while-loop has finish evaluating the condition.

Pre-2.7 API

This section covers the API that was used before v2.7. New code should try to use the latest API since the code is easier to read and has more compile-time validation.

template <typename T>
T get(LockType const& lock, size_t offset) const

Returns a value of type T from memory. The register offset is given by the function parameter offset. If the offset would result in access outside the memory space, this method will throw a std::range_error exception. This function requires a LockType to be passed to prove you have exclusive access to the hardware.

If the offset is known at compile-time, it is much more efficient to call get<LockType, size_t>() since this function requires a run-time check every time it's called.

template <typename T, size_t offset>
T set(LockType const& lock, T const value) const

Sets a register of type T to the value, value. The register offset is given by the template parameter offset. This function requires a LockType to be passed to prove you have exclusive access to the hardware.

template <typename T>
T set(LockType const& lock, size_t offset, T const value) const

Sets a register of type T to the value, value. The register offset is given by the function parameter offset. If the offset would result in access outside the memory space, this method will throw a std::range_error exception. This function requires a LockType to be passed to prove you have exclusive access to the hardware.

If the offset is known at compile-time, it is much more efficient to call get<LockType, size_t>() since this function requires a run-time check every time it's called.

Planned Improvements

There should be a way to incorporate DMA transfers in the API.
The constructor ought to do a vxMemProbe() to make sure the address range is usable.

Example

We define a class that uses A16-based hardware:

 1 using namespace vwpp::v2_7;
 2
 3 class Card {
 4     Mutex mutex;
 5
 6     typedef Mutex::PMLock<Card, &Card::mutex> LockType;
 7     typedef VME::Memory<VME::A16, VME::D16_D32, 0x100, LockType> HW;
 8
 9     typedef VME::Register<VME::A16, uint32_t, 0xfc, VME::Read, VME::Write> regA;
10     typedef VME::Register<VME::A16, uint16_t, 0x0, VME::Read, VME::NoWrite> regB;
11
12     HW hw;
13
14  public:
15     Card(uint16_t const addr) : hw(addr) {}
16
17     inline uint16_t setReadHardware(uint32_t const v)
18     {
19         LockType lock(this);
20
21         hw.set<regA>(lock, v);
22         return hw.get<regB>(lock);
23     }
24 };

(line 4) Each instance of the object will have its own, private mutex to serialize access to the hardware.
(line 6) A typedef is defined to make the code easier to read. The typedef defines LockType to be a lock that manages ownership of a mutex in the Card class.
(lines 9 and 10) Hardware registers are defined using register "traits".
(line 12) A Memory object is declared. The hardware takes up 256 bytes of the space (0x100) and it's going to be serialized using the LockType we defined earlier.
(line 15) The constructor passes the A16 offset to the constructor of the Memory object.
(lines 17-23) We define a method, setReadHardware(), which atomically sets and reads the hardware. The get and set methods use the register traits to determine how and where in the memory bank the data is accessed.

All hardware access is verified at compile time (try compiling it after changing an offset to something greater than 0x100.) The extra wordiness is necessary to prove to the compiler that you're meeting all the conditions, but the generated code is comparable to hand-written code. If we generate assembly, we get[^1]:

    ; hw.set<regA>(lock, 10)

    lwz     9,16(1)         ; r9 is loaded with the hardware's base address
    eieio
    li      0,10            ; r0 is loaded with 10
    stw     0,252(9)        ; 32-bits of r0 is stored in r9 + 252 bytes

    ; return hw.get<regB>(lock)

    lwz     11,16(1)        ; r11 is loaded with the hardware's base address
    eieio
    lhz     29,0(11)        ; r29 is loaded with 16-bits from r11 + 0 to be returned

C++ Template Syntax

Examples given so far use a globally or locally defined instance of a memory space. The syntax for using registers in this case is fairly elegant, in the author's opinion. If a memory space template is used in a templated class, the code gets a little messy due to ambiguities in C++'s complex syntax. For instance, let's take the previous example and change the class to be a template. Instead of having an internal mutex, the template parameter will be the lock type used for serialization.

 1 using namespace vwpp::v2_7;
 2
 3 template <typename L>
 4 class Card {
 5     typedef typename DetermineLock<L>::type LockType;
 6     typedef VME::Memory<VME::A16, VME::D16_D32, 0x100, LockType> HW;
 7
 8     typedef VME::Register<VME::A16, uint32_t, 0xfc, VME::Read, VME::Write> regA;
 9     typedef VME::Register<VME::A16, uint16_t, 0x0, VME::Read, VME::NoWrite> regB;
10
11     HW hw;
12
13  public:
14     Card(uint16_t const addr) : hw(addr) {}
15
16     inline uint16_t setReadHardware(LockType const& lock, uint32_t const v)
17     {
18         hw.template set<regA>(lock, v);
19         return hw.template get<regB>(lock);
20     }
21 };

(line 3) Card is now a template class and the parameter is a lock type.
(line 5) Uses a helper template from the VWPP library which verifies type L is a lock type.
(line 16) Instead of making the lock, the class's method now requires a valid lock to be given.
(lines 18 and 19) Because we're using templates within templates, C++ requires this ugly addition in order to understand what we're trying to do.

Defining Registers Correctly

If you've ever looked at the assembly language generated when using Wind River's sysIn() and sysOut() functions, you saw the large number of instructions it uses to implement them. This library tries to reduce the instructions it takes to access the hardware to the bare minimum and succeeds; instructions that access the hardware are separated by very few "glue" instructions. This optimization has a caveat, though: since so few instructions are generated, the sequence is sensitive to the optimizer re-ordering instructions to improve pipelining and to the CPU's instruction decoder which is allowed to pre-fetch memory -- even if a condition branches away from the read. To ensure correct behavior, you will need to define the registers appropriately.

As an example, suppose we have hardware with a status register (offset 0) indicating whether a FIFO has content and a register (offset 2) which reads and removes an entry from the FIFO. Using this library, a naive implementation might do this:

 1 Lock lock;
 2
 3 typedef Register<VME::A16, uint16_t, 0, Read, NoWrite> regStatus;
 4 typedef Register<VME::A16, uint16_t, 2, Read, NoWrite> regFifo;
 5
 6 while ((a16.get<regStatus>(lock) & 0x1) != 0) {
 7     uint16_t item = a16.get<regFifo>(lock);
 8
 9     // Do something with the FIFO item.
10 }

(line 6) This loop continues while a status bit indicates there's data in the FIFO. If there isn't any, then the loop stops.
(line 7) This reads the oldest item in the FIFO. Due to optimizations, there's only a few instructions separating this access from the previous test. The PowerPC is allowed to pre-fetch reads from memory if the memory system isn't being used by earlier instructions in the queue. This means there's a tiny chance that the status register indicated no items were in the FIFO yet, when the instruction prefetch reached this line, there were items. This would appear as the FIFO dropping data once in a great while.

The solution is to mark the FIFO register as DestructiveRead. This makes the template generate code that forces the processor to complete all previous pending read and writes and preloaded instructions before continuing. In this example, it'll continue only if the while-loop's condition indicated there was something in the FIFO.

 1 Lock lock;
 2
 3 typedef Register<VME::A16, uint16_t, 0, Read, NoWrite> regStatus;
 4 typedef Register<VME::A16, uint16_t, 2, DestructiveRead, NoWrite> regFifo;
 5
 6 while ((a16.get<regStatus>(lock) & 0x1) != 0) {
 7     uint16_t item = a16.get<regFifo>(lock);
 8
 9     // Do something with the FIFO item.
10 }

The rule-of-thumb is, if reading a register has a side-effect, the register should specify DestructiveRead.

[^1] I reordered the instructions from the assembly output so it would better match the source. The compiler's original sequence had all these instructions, but were ordered to keep all the CPU subsystems active.

Navigation Menu

Uh oh!

Memory Spaces

VME Memory Class

`VME::Memory` API

Constructor

Copy Constructor

Future enhancement

`VME::Register` Class

`VME::Register` API

Pre-2.7 API

Planned Improvements

Example

C++ Template Syntax

Defining Registers Correctly

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Uh oh!

Memory Spaces

VME Memory Class

VME::Memory API

Constructor

Copy Constructor

Future enhancement

VME::Register Class

VME::Register API

Pre-2.7 API

Planned Improvements

Example

C++ Template Syntax

Defining Registers Correctly

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

`VME::Memory` API

`VME::Register` Class

`VME::Register` API