Overview

Overview
	Chapter 362. TILE-Gx TMC Library

Name

Overview — eCos Port of a Subset of the TMC Library

Description

The Tilera Multicore Components or TMC Library provides a variety of primitives for building parallel programs. It is documented in UG527, “The Applications Libraries Reference Manual”. The eCos package CYGPKG_HAL_TILEGX_TMC implements a subset of this library. The subset supports communication between Linux applications running on some of the tiles on a TILE-Gx chip and eCos applications running on other tiles. The package also contains a number of example applications demonstrating the communication functionality.

The package CYGPKG_HAL_TILEGX_TMC is automatically included in any configuration for a TILE-Gx target. It does not have to be added to the configuration. The package does not add any overhead to eCos applications which do not use any of its functionality, so there is no reason for ever removing the package from the configuration.

The Tilera TMC library consists of the following components:

UDN helper routines for communication over the UDN bus. Most of these have been ported to eCos.
Performance tuning. The routine tmc_perf_get_cpu_speed() is implemented.
Spinning shared memory synchronization primitives. These have all been ported to eCos.
Scheduler shared memory synchronization primitives. These have not been ported. The primitives interact with the Linux kernel, and there is no way for an eCos application running on a BME tile to do that.
Specified-attribute memory page allocation. These have not been ported. Allocating memory pages involves calling into the hypervisor and the hypervisor is no longer present on BME tiles running eCos. However it is possible for a Linux application to allocate one or more pages, pass the details on to an eCos application, and have the latter map the pages into its address space.
Common memory, allowing pages to be mapped at the same virtual address in different processes. These have not been ported. Linux applications run in 64-bit mode with a 64-bit address space, whereas eCos applications run in 32-bit mode with 32-bit addresses. This makes it difficult to use the same virtual addresses.
CPU sets and affinitization. These have not been ported. The primitives are intended to allow threads to be bound to specific tiles. Since each eCos instance runs on only one tile there is no point in attempting to support such affinitization.
User space interrupt installation routines. These have not been ported. eCos has its own model of how interrupts should be handled and its own routines for managing interrupts. Trying to support the TMC's model of interrupt handling as well would complicate things for little or no gain.
Interprocessor interrupt event-handling. These have not been ported. It is not clear that they are actually useful since there is no primitive for generating an IPI interrupt. Instead UDN communications provides a way of sending data asynchronously to another tile, and if desired that UDN communication can be processed by an interrupt handler.
Using mspaces for standard malloc/free. These have not been ported. Under Linux they provide an alternative implementation of the C library's malloc() and free() routines, offering some control over home caching and memory page sizes. For eCos applications it makes more sense to use the standard eCos heap using memory provided by the hypervisor during startup.
Cache control. These functions have not been ported, and the Tilera documentation recommends against using these low-level shared memory primitives. The main functionality provided, memory fences to guarantee visibility of stores to cache coherent memory, is instead provided by the eCos HAL_MEMORY_BARRIER() macro.
Multiple heap allocation. These routines allow for the allocation of separate mspaces with control over cache homing and other functionality. They involve interaction with the hypervisor's memory page support which is not possible for an eCos application.
Task management and cleanup. These primitives relate to support for multiple processes and interaction with the Tilera shepherd process. Since eCos does not support multiple processes, only multiple threads, the primitives are not applicable.

The package only provides implements of the header files <tmc/udn.h>, <tmc/spin.h>, and <tmc/perf.h>. The other header files do exist but will generate a compile-time warning if they are included. Providing these dummy headers prevents the compiler from accidentally including the Linux TMC headers.

Restrictions

The eCos TMC support is subject to a number of important restrictions which application developers must be aware of.

When a Linux process uses tmc_alloc_map() or a similar routine to allocate a block of memory, that memory is owned by the Linux process. Details of the block including the physical address can be passed on to eCos applications which can then map it into their address space. If the Linux process exits or gets killed off the Linux kernel and hypervisor will reclaim the allocated block, which may then get reused for some other Linux process or for the kernel or hypervisor itself. Meanwhile the eCos application may still have a mapping to the underlying physical memory and may still write to it, corrupting memory that now belongs to some random other part of the system. Neither the hypervisor nor the Linux kernel have any way of keeping track of what memory has been mapped into an eCos application's address space, so they cannot do anything to avoid this problem.

The only solution is to make sure that the eCos application is always informed when the Linux process exits, so that it can unmap any shared memory pages. That is not always easy to achieve, especially in a debug environment, but it is the application developer's responsibility.

Separately, UDN communication is subject to a major restriction. By default a Linux application does not have access to the UDN bus, and must explicitly obtain such access from the Linux kernel by a call to tmc_udn_init(). The kernel only grants UDN access to one task per tile. In a debug environment LittleBoPeep runs on one of the tiles providing gdb debug functionality for eCos tiles, and LittleBoPeep needs to use the UDN bus for this. Therefore Linux applications requiring UDN access cannot run on the same tile as LittleBoPeep, usually the highest-numbered tile not used for BME.

UDN Support

The following UDN routines are supported:

tmc_udn_header_from_cpu()

tmc_udn_send_buffer()

tmc_udn0_receive_buffer(), tmc_udn1_receive_buffer() and tmc_udn2_receive_buffer()

tmc_udn_send()

tmc_udn_send_1() to tmc_udn_send_20()

tmc_udn0_receive() to tmc_udn2_receive()

tmc_udn0_available_count() to tmc_udn2_available_count()

tmc_udn_available_mask()

The functions related to UDN channel 3, tmc_udn3_receive_buffer(), tmc_udn3_receive() and tmc_udn3_available_count() are not supported. UDN channel 3 is used for communication between LittleBoPeep and gdbstubs, and if application code tried to use this channel as well then things would get very confusing with UDN traffic going to the wrong program.

There are four other UDN functions in the Tilera TMC library which are not supported under eCos: tmc_udn_init(), tmc_udn_close(), tmc_udn_activate(), and tmc_udn_persist_after_exec(). Under Linux tmc_udn_init() is needed to request access to the UDN bus from the Linux kernel. That is not necessary under eCos since eCos applications run at protection level 2, which is sufficient for UDN access. The three other functions are also related to access rights and are equally unnecessary.

Spinning Shared Memory Synchronization

The following are supported:

tmc_spin_mutex_t

`TMC_SPIN_MUTEX_INIT`	`tmc_spin_mutex_init()`
`tmc_spin_mutex_lock()`	`tmc_spin_mutex_trylock()`
`tmc_spin_mutex_unlock()`

tmc_spin_queued_mutex_t

`TMC_SPIN_QUEUED_MUTEX_INIT`	`tmc_spin_queued_mutex_init()`
`tmc_spin_queued_mutex_lock()`	`tmc_spin_queued_mutex_trylock()`
`tmc_spin_queued_mutex_unlock()`

tmc_spin_rwlock_t

`TMC_SPIN_RWLOCK_INIT`	`tmc_spin_rwlock_init()`
`tmc_spin_rwlock_rdlock()`	`tmc_spin_rwlock_wrlock()`
`tmc_spin_rwlock_tryrdlock()`	`tmc_spin_rwlock_trywrlock()`
`tmc_spin_rwlock_rdunlock()`	`tmc_spin_rwlock_wrunlock()`
`tmc_spin_rwlock_unlock()`

tmc_spin_barrier_t

`TMC_SPIN_BARRIER_INIT()`	`tmc_spin_barrier_init()`
`tmc_spin_barrier_wait()`

These data types and functions have the same semantics as the Tilera TMC versions, so the Tilera documentation can be consulted for more details.

An important point about these routines is that the various unlock functions and the barrier wait function involve a memory barrier, guaranteeing that all memory writes are visible to other tiles. Therefore code that only manipulates shared data while owning a lock automatically avoids many memory consistency problems.

Example Applications

The package comes with a number of example applications in the examples subdirectory:

udn0: This example illustrates UDN communication between a TILE-Gx Linux application and one or more instances of an eCos application.
udn1: This example is derived from udn0. It adds direct communication between the instances of the eCos application.
shm0: This example sets up a block of shared memory between a Linux application and one or more instances of an eCos application. The shared memory is used to hold large amounts of data. The Linux application sends UDN messages to control what each eCos instance does with that shared data.
shm1: This example also sets up a block of shared memory. However UDN communication is used only during initialization, to set up the shared memory. All subsequent communication between Linux and eCos goes via the shared memory.
spintest: This is a testcase for the various spinning shared memory synchronization primitives.
not61850: This is an example involving a Linux host application, a TILE-GX Linux application, and one or more instances of an eCos worker application. It combines ethernet traffic using the gxio/mpipe library routines running in the TILE-Gx Linux application, and communication between that and the eCos workers over shared memory and the UDN bus. It was written to a specific customer's requirements and may be of limited interest to other users.


Chapter 362. TILE-Gx TMC Library		Real-time characterization of selected targets

2024-12-10

eCosPro Non-Commercial Public License