# FoundationDB Authors - Simulation and Testing (Highlights)

## Metadata
**Cover**:: https://readwise-assets.s3.amazonaws.com/static/images/article2.74d541386bbf.png
**Source**:: #from/readwise
**Zettel**:: #zettel/fleeting
**Status**:: #x
**Authors**:: [[FoundationDB Authors]]
**Full Title**:: Simulation and Testing
**Category**:: #articles #readwise/articles
**Category Icon**:: 📰
**Document Tags**:: #distributed-system #test
**URL**:: [apple.github.io](https://apple.github.io/foundationdb/testing.html)
**Host**:: [[apple.github.io]]
**Highlighted**:: [[2020-04-16]]
**Created**:: [[2022-09-26]]
## Highlights
- robust simulation, live performance testing, and hardware-based failure testing.
- increased intensity of the failures
- a *deterministic* simulation of an entire FoundationDB cluster within a single-threaded process
- Determinism is crucial in that it allows perfect repeatability
- simulates all physical components
- To swizzle-clog, you first pick a random subset of nodes in the cluster. Then, you “clog” (stop) each of their network connections one by one over a few seconds. Finally, you unclog them in a random order, again one by one, until they are all up.