Hydra Speculation Hardware Modified Bit Pre-invalidate Bit Read
Bits Write Bits
The goal of any high-performance architecture is to exploit
program parallelism. Now parallelism exists at multiple levels. At
the lowest level is the parallelism between instructions in a
basic-block, Above that is the loop-level parallelism between loop
iterations. Above this is thread-level parallelism that comes from
parallelizing a single application either manually or
automatically. At the highest level is the parallelism between
separate processes in a multiprogramming environment. The
granularity of parallelism typically increases as the level of
parallelism increases. Superscalar architectures concentrate on
exploiting ILP and to some extent LLP, but these techniques are
complex and do not scale well in advanced semiconductor
technologies. So this begs the question
*The approach we have taken in Hydra is multiprocessor built out
of simple processors that can be designed for high speed. The main
advantage that this architecture is multiple threads of control
which enables parallelism to be exploited that is much more widely
distributed in the program than is possible with a single hardware
window. Secondly, if each of the processors is tightly integrated
with its first level cache the highest speed paths can be localized
to reduce the impact of wire delay and make it possible to achieve
very high clock frequencies. The disadvantage of this architecture
is that it requires parallel programs to run on more than one
processor. We use thread level speculation to simplify the process
of creating parallel programs. Our overall design approach in Hydra
is to keep the basic MP architecture as simple as possible by
taking advantage of the fact that we are on single piece of
silicon. Well see how we do this.*The problem is that the amount
parallel software is limited. Today, most parallel software is
generated manually. This is typically a long and difficult process.
Traditional parallelizing compilers can help but they only work
well for dense matrix FORTRAN programs and so are not a solution
for parallelizing general purpose programs particularly C programs.
The problem with C programs is figuring out the memory dependencies
and placing synchronization to honor true-data dependencies. This
is especially a problem where pointers are used. Typically the
pointer disambiguation algorithms are not powerful enough and so
limit the amount of parallelism that can be found. Furthermore the
compiler must be conservative. So that if 99% of the iterations of
a loop are parallel but 1% are not the loop is declared as not
parallel.
We would like to change this situation by adding hardware to
allow the compiler to be aggressive rather than conservative.The
technique we will use to allow the compiler to be aggressive is
called data-speculation. Data speculation allows sequential
programs to execute in parallel, but ensures that all loads and
stores follow the sequential order so that program correctness is
maintained. Now synchronization between parallel threads is only
needed for performance and loops can be paralllelized almost
obliviously.
Loop iterations are not the only program construct that can be
parallelized in this way. All that is required is a sequential
execution order to the threads. Another construct that we have
experimented with is procedures where the code after the procedure
is executed in parallel with the procedureSo what sort of hardware
support do you need for data speculation.
The basic functionality we would like to provide is memory
location renaming which is analogous to register renaming in
dynamically scheduled processors. Lets suppose we have two
iterations of a loop (I and I +1). Naturally a write to x in the
earlier iteration will be read by a read of x in the later
iteration. Let us suppose that these iterations are executed in
parallel. Now if the write occurs before the read we would like the
value of x to be forwarded to the latter iteration, but if the read
occurs before the write we have a violation we want he memory
system to detect this case and signal the violation.The simplest
although not the most efficient way of recovering from a violation
is to re-execute the loop iteration
However, if this loop-iteration is re-executed, the state of the
memory system should be the same as when the loop first executed.
In particular all writes from the violated iteration must be
discarded and must not be allowed to change the sequential
permanent state of the machine. On the other hand, if iteration I+1
finishes without violations all writes must change the permanents
state after writes from iteration I. Note that writes to the same
variable get renamed [memory location renaming].Also note that
iteration I can cause violations in iteration I+1 even if it
finishes after I+1. In other words all threads must commit in-order
even though they execute in parallel.Forward progress is always
maintained because one thread (running on head processor) will
always execute non-speculatively, and there will be immune from
violations. Limited buffer size: Since we need to buffer state from
a speculative region until it commits, threads need to be short
enough to avoid filling up the buffer space allocated for data
speculation too often. An occasional full buffer can be handled by
simply stalling the thread that is producing too much state until
it becomes the head processor, when it may continue to execute
while writing directly to memory. However, if this occurs too
often, performance will suffer. True dependencies: Excessively
large threads have a higher probability of dependencies with later
threads, simply because they issue more loads and stores. With more
true dependencies, more violations and restarts occur. Restart
length: A late restart on a large thread will cause much more work
to be discarded, since a checkpoint of the system state is only
taken at the beginning of each thread. Shorter threads result in
more frequent checkpoints and thus more efficient restarts.
Overhead: Very small threads are also inefficient, because there is
inevitably some overhead incurred during thread creation and
completion. Programs that are broken up into larger numbers of
threads will waste more time on these overheads.*More threads than
processors.
Coprocessor support: Table of exception vectors for speculation
vectorsHandled by software routines as well as exceptional handlers
triggered by hardware eventsContains timers and prediction tables
used to prevent speculation on nonparallel threads and predict
return values for speculative
L1 CACHE InfoModified Bit: This bit acts like a dirty bit in a
writeback cache.If any changes are written to the line during
speculation, this bitis set. These changes may come from stores by
this processor orbecause a line is read in that includes
speculative data fromactive secondary cache buffers. If a thread
needs to be restartedon this processor, then all lines with the
modified bit set aregang-invalidated at once.
Pre-invalidate Bit: This optional bit is set whenever another
processorwrites to the line, but is running a more
speculativethread than this processor. Since writes are only
propagatedback to more speculative processors, we are able to
safely delayinvalidating the line until a different, more
speculative thread isassigned to this processor. Thus, this bit
acts as the opposite ofthe modified bit it invalidates its cache
line when the processorcompletes a thread. Again, all lines must be
designed forgang-invalidation. If pre-invalidate bits are not
included, writesfrom more speculative processors must invalidate
the lineimmediately to ensure correct program execution
Read Bits: These bits are set whenever the processor reads froma
word within the cache line, unless that words written bit isset. If
a write from a less speculative thread, seen on the writebus, hits
an address in a data cache with a set read bit, then atrue
dependence violation has occurred between the two processors.The
data cache then notifies the processors CP2, initiatinga violation
exception. Subsequent stores will not activate thewritten bit for
this line, since the potential for a violation hasbeen
established.
Write Bits: To prevent unnecessary violations, this bit or set
ofbits may be added to allow renaming of memory addresses usedby
multiple threads in different ways. If a processor writes to
anentire word, then the written bit is set, indicating that this
threadnow has a locally generated version of the address.
Subsequentloads will not set any read bit(s) for this section of
the cacheline, and therefore cannot cause violations.
What happens on a L1 cache conflict?Normally flush to L2 if read
bits set, then the processor must halt execution until it becomes
the head processor because otherwise no way to detect speculation
violations in L2. Can have a victim cache for this as well3
speculative CPUs and one non-specualtive head CPU.Larger
speculative miss rates during speculation inter processor
communication misses in speculation and restarts with L2 loads%
increase of loads so high because superfluous speculative memory
accesses which are then restartedCompress() is an anomaly b/c
uniprocessor optimized code had more register saves across function
calls than the speculative loop body*
