Lesson 6: Shared Memory and Cache Coherence
- discussion thread
-
^ Primer on Memory Consistency and Cache Coherence
Chapters 2,6,7,8 -
* A low-overhead coherence solution for multiprocessors with private cache memories
Papamarcos et al., ISCA '84 - tasks due January 27
On readings: Recommended background readings are marked with (^) above. Optional historical or fun readings are marked with (*). If you feel confortable with the topic already, you may skip these readings.
Notes
I will cover a few key ideas here, but the slides from lecture can be found on Canvas.
Key Terms
- ccNUMA: cache-coherent NUMA
- SMP: Symmetric multi-processor (contrast to NUMA)
- ping-ponging: when a cache line bounces back and forth over the coherence interconnect, creating performance issues. Typical with locks
- Reader-writer lock: a scalable lock implementation that differentiates between threads trying to read and those trying to write
- RPC: Remote procedure call. See Ch. 48.5 of here
- TLB shootdown: When a multi-threaded process running across cores has any change to its address space (i.e., a page table entry is changed, added, or removed), the TLB must be flushed. But we can’t just flush our local core’s TLB now. We have to tell other cores to do the same, otherwise threads might see different views of the address space.
- IPI: Inter-processor interrupt. Hardware mechanism to send an asynchronous interrupt message to another CPU core. These are used for several OS events, a notable one being TLB shootdowns.
- microkernel: OS architecture that pushes most functionality out to userspace. Microkernels use a modular structure where userspace OS components communicate with one another using inter-process communication (IPC) mechanisms, mediated by the core (privileged) kernel.
- two-phase commit: (2PC) a protocol to commit distributed transactions (e.g. a write to multiple replicas). An originator process/thread sends a “prepare” message to all participants. If all acknowledge, the originator process sends a commit message, and all participants commit the transaction. If >= 1 participants fail to prepare or reject the prepare message, the entire transaction is rolled back.
Trends
Think about the trends used as motivation in the paper (which is more than a decade old now). What trends have continued?
Distributed Systems
We could have an entire course about agreement protocols and distributed algorithms, but one thing to think about as we cover the basics of consensus and state machine replication later on is: do their claims align with the issue of scale?
Follow-up Reading
Some of the core ideas from Barrelfish are still being explored in different ways. See