TITLE: DESIGN ISSUES AND FUTURE TRENDS OF DISTRIBUTED SHARED MEMORY SYSTEMS
ABSTRACT
In these times, the distributed shared memory paradigm has gained a lot of attention in the field of distributed systems. This piece of work looks into different system issues that arise in the design of distributive shared memory systems. The work has been motivated by the observation that distributed systems will continue to become popular and will be largely be used to solve large computational issues. Since shared memory paradigm offers a natural transition for a programmer from the field of uniprocessors, it is very attractive for programming large distributed systems.
Introduction
The motive of this research is to identify a set of system issues, such as integration of DSM with virtual memory management, choice of memory model, choice of coherence protocol, and technology factors; and evaluate the effects of the design alternatives on the performance of DSM systems. The design alternatives have been evaluated in three steps. First, we do a detailed performance study of a distributed shared memory implementation on the CLOUDS distributed operating system. Second, we implement and analyze the performance of several applications on a distributed shared memory system. Third, the system issues that could not be evaluated via the experimental study are evaluated using a simulation-based approach. The simulation model is developed from our experience with the CLOUDS distributed system. A new workload mode that captures the salient features of parallel and distributed programs is developed and used to drive the stimulator. The key results of the thesis are: DSM mechanisms have to be integrated with the virtual memory management for providing high performance distributed shared memory systems; the choice of the memory model and coherence protocol does not significantly influence the system performance for applications exhibiting high computation granularity and low state-sharing; and an efficient implementation of DSM requires a careful design of miscellaneous system services (such as synchronization and data servers). The thesis also enumerates several questions related to future research directions.
Issues in the design of distributed shared memory systems.
1. Virtual memory and distributed shared memory system

Since DSM is not true shared memory as in shared memory multiprocessors, remote memory accesses have to be reconciled with the memory management at each node. If the basic machine does not support virtual memory then the solution could be simpler. However, if the basic architecture supports virtual memory than the DSM management and virtual memory management have to be integrated. The local memory at each node may be considered simply a cache of a global address space that spans the entire network. The DSM and virtual memory management at each node would have to cooperate to ensure that the semantics implemented by the DSM manager and the virtual manager not compromised. The effeteness of the DSM paradigm depends critically on how quickly a remote memory access request is serviced and the computation is allowed to continue, which in turn depends on these factors.
• The speed at which the virtual manager system detects that a memory access fault or a pre-fetching request entails a remote access.
• The software overhead involved in the DSM protocol for servicing a remote memory access request.
• The software overhead involved in the communication subsystem for effecting the inter-node message communication to service the request.
• The speed of the communication medium.

2. Granularity
The two dimensions to granularity include computation granularity and data granularity. Computation granularity deals with the amount of computation a process has to do between synchronization or communication points in a multi-process computation. Data granularity deals with the amount of shared information processed during this computation phase.
Write-runs is defined as a sequence of read and write by a given processor following an initial write executed by the same processor to a given shared memory location before an external read by a different processor to that shared memory location occurs. In shared memory multiprocessor system, write-runs of representative applications may range from a few to a few tens of references. However, in a DSM system, write-runs of a few hundred instructions would be more appropriate given the latency for remote accesses.
Another distinction between the SMM and DSM is in the data granularities of accesses that are practical in the two. In a uniprocessor memory hierarchy, the processor to cache transfer time is in the tens of nanoseconds, the cache-to-main memory transfer time is in the hundreds of nanoseconds and the main memory to disk transfer time is in the order of milliseconds. DSM systems add a new dimension to the memory hierarchy namely remote memory access across the network. The choice of the network plays a big role in determining the latency. All the same, independent of this choice, there is a fixed software overhead to be incurred depending on the choice of the data transfer protocol on the network. Moreover, such remote memory accesses need to be integrated somehow with the memory management at each node. This requirement often forces the granularity of access to be an integral multiple of the fundamental unit of memory management. It is however possible to reduce the latency by transferring the page partially. Data granularity has to be sufficiently high to make the DSM paradigm viable.
3. Memory model
In a uniprocessor, correctness of execution is ensured by preserving the order of memory references generated by a processor. As far as correctness of multiprocessor execution is concerned only the ordering of the shared memory references is of interest. Shared memory accesses are categorized into three types:
• Shared code
• Synchronization variables
• Shared data variables
While shared code is always read only, synchronization variables requires that memory consistency be strictly preserved. Shared data variables normally require strict consistency as well. Several applications exist where in the program correctness would not be compromised even if there are temporary inconsistencies in the view of shared memory as experienced in distinct processed. Weak ordering, an alternative to sequential consistency requires memory access from a process is performed in a program order; synchronization operations are globally performed before allowing a process to continue; and all shared data access from a process are globally performed before issuing a synchronization operation.
4. Choice of protocol
DSM assumes that all memory is globally shared. This assumption requires independent computations started at different nodes see a consistent view of the shared memory. Multiprocessor cache consistency protocols ensure the following memory coherent constraints: a read operation performed by a processor returns the most recent value written into that location.